# Performance Benchmarks of S3 Storage Middleware

- Institution: CERN
- Speaker: Huseyn Gurbanov
- Year: 2026
- Language: English
- Subjects: Computer science
- Source: [CERN](https://videos.cern.ch/record/3025480)
- Licence: [CERN](https://copyright.web.cern)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Amazon's Simple Storage Service (S3) protocol has become a de-facto standard for object storage, and academic supercomputing centres are increasingly interested in offering S3-compatible storage solutions. This is on the one hand due to more (on-premises) cloud computing at such centres, and on the other hand due to the increasing demands of flexible data access and sharing across computing sites. Certainly, the movement towards FAIR (Findable, Accessible, Interoperable, Reusable) Research Data Management plays a significant role here.

The EXA4MIND Horizon Europe project aims at offering optimised storage back-ends, including object storage and databases, for Extreme-Data applications at European supercomputing centres. As part of this effort, we have conducted performance tests of different open-source middleware alternatives to implement S3 storage. We have performed these benchmarking experiments on bare-metal, virtual-machine and High-Performance Computing (HPC) systems.

We show first results using widely-used open-source S3 middleware such as MinIO, S3Proxy, and VersityGW. It appears that a combination of such middleware and optimised client software, making use of multiple parallel streams, is able to saturate network bandwidths up to several GB/s, possibly enabling direct object-store usage from HPC codes. Typical performance-limiting factors include I/O based on small files in the 100 MB range or below, and a too strongly restricted number of streams.

Acknowledgement: This work received support from the EXA4MIND project ("EXtreme Analytics for MINing Data spaces"), funded by the European Union's Horizon Europe Research and Innovation Programme, under Grant Agreement N° 101092944. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the granting authority can be held responsible for them.

00:00:00 Slide 1

00:00:06 Slide…

## Lectures

1. [Performance Benchmarks of S3 Storage Middleware](https://invisiblenetwork.ai/learn/cern/3025480/ddb5e4906868), 11 min (CERN's player)