About
In our talk at CS3 2025 we will discuss the (r)evolution in the storage and network back-ends for cloud-based data centric services with a special focus on the sync & share service as a storage killer application.
We will review various storage back-ends for cloud platforms along with their performance, scalability and maintenance complexity and economical aspects. We will also touch on the new trends in the cloud platforms that convert into hyper-convergent, fully software-defined and open source-based systems, where compute, storage and network components act together to provide flexible, functional, scalable, reliable and high-performance platform that is cost-effective, easy to implement, maintain, automate, monitor and optimize.
In face of the AI revolution and overall shift of economy, science, research and education towards data-centric applications and services we observe increasing requirements vs cloud platforms that help dealing with large data sets. At the very infrastructure level this means a growing pressure on performance (IOPS, GB/s) and economical scalability. We can also observe that 'classical' storage technologies: disk arrays, Fibre Channel-based SAN networks etc. become obsolete and legacy as they lack flexibility, do not support programmatic orchestration widely, and are relatively costly to purchase, implement and maintain. Their usage requires specialised knowledge and can lead to vendor lock-in situations. On the other end of technology spectrum, commodity technologies are rapidly developed, breaking the next barriers of reliability and performance. Growing popularity and improving economic affordability of flash storage systems (SSD, NVMe) enables advancing the cloud platforms and applications performance so that they address today's IO requirements. Also recent development of Ethernet enables using this protocol as a carrier for I/O traffic. RoCE is in particular useful to implement RDMA in Ethernet networks that are widely spread in…
The talk
Credits and licence
- Institution
- CERN
- Department
- HEP Computing
- Speaker
- Krzysztof Wadówka
- Year
- 2025
- Language
- English
- Subjects
- Computer science
- Source
- CERN
- Licence
- CERN, as CERN states it. CERN's terms
- Attribution
- © CERN. CERN provides its audiovisual media free of charge for educational and informational use, credited to CERN; this lecture is shown in CERN's own player.
- Who can watch
- CERN gives no rating. Invisible Network shows it to Teens profiles and older, judged from what the source says about it. How courses are placed