# Comparison of High-Performance Distributed File Systems on two Platforms: Linux and Windows

- Institution: CERN
- Department: HEP Computing
- Speaker: Gregor Molan
- Year: 2024
- Language: English
- Subjects: Computer science
- Source: [CERN](https://videos.cern.ch/record/3004607)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Subtitle: Superiority of EOS-based Comtrade Distributed File System (CDFS) for Earth Observation Data Storage.

Introduction ------------ The rising quantity of data collected necessitates transitioning to the next generation of reliable, high-performance data storage solutions. Despite the clear need for high-performance storage across many industries, there has yet to be a consensus on the optimal high-performance storage technology that would suit all user needs.

The project's goal was speed-testing to provide a possibility to combine them with the currently available state-of-the-art technology results, technological capabilities and existing expertise.

Methodological approach ----------------------- We choose representative high-performance storage solutions compared on two different platforms for storage clients.

- Linux: - Ceph - EOS - IBM Spectrum Scale - Hadoop - Windows: - Ceph - EOS-drive - EOS through Samba - EOS-wnc - Hadoop

An example of specifications of the testing environment is the following.

- *Management node:* - 32 threads - 2x Intel® Xeon® Silver 4208 Processor - 384 GB of RAM - 2x SSDs of 2 TB. - *Storage nodes:* - 32 threads - 2x Intel® Xeon® Silver 4208 Processor - 64 GB of RAM - 1x SSD of 2 TB for the operation system - 6x HDDs of 2 TB for data. - Client nodes: - 12 threads - 1x Intel® Core™ i5-12400 Processor - 16 GB of RAM - 1x SSD with 1 TB.

Results ------- The results are from the testing performed separately in an isolated environment for each high-performance solution but on the same hardware. For each platform, there were three categories of tests related to file sizes: (1) small, (2) medium, and (3) large files.

According to the results, some high-performance file systems have evident advantages, as shown in our presentation. These results should be the starting point for an even more exact comparison between these file systems. They are good starting points in choosing the right high-performance file system..

0:00:00…

## Lectures

1. [Comparison of High-Performance Distributed File Systems on two Platforms: Linux and Windows](https://invisiblenetwork.ai/learn/cern/3004607/9cf541e36aef), 12 min (CERN's player)