Distributed data sharing infrastructure for scientific institute based on Onedata

About

CEITEC, a research institute established at Masaryk University, hosts multiple core facilities and research groups that jointly produce large volumes of heterogeneous scientific data across diverse scientific domains. These activities require systematic recording of experimental outputs and their timely delivery to researchers for analysis and interpretation.

A key requirement is controlled and reliable data handover. Ad-hoc transfers via USB drives or personal workstations lead to data fragmentation, loss of provenance, and limited reproducibility, and therefore must be replaced by centralized storage on institutional infrastructure. At the same time, users expect an access model comparable to contemporary cloud storage services such as Google Drive or Dropbox, combined with seamless integration into institutional compute environments. Keeping data close to computational resources—such as Kubernetes clusters, Jupyter notebooks, or Galaxy workflows—eliminates repeated manual copying and significantly lowers the barrier to data analysis.

Certain data-intensive experiments, for example in cryo-electron microscopy, impose additional constraints. Measured data must be made available to users while the acquisition is still in progress, enabling real-time inspection and timely adjustment of instrument parameters. This combination of real-time access, controlled sharing, and tight coupling to compute infrastructure creates strict requirements on data management and sharing systems.

To address these challenges, we adopted the Onedata platform as the foundation of a distributed data sharing infrastructure. Onedata provides unified access, controlled sharing, and transparent integration of distributed storage resources while preserving data locality and performance. This contribution presents the concrete workflows deployed at CEITEC, describes their integration into everyday laboratory and computational practices, and reports operational experience from running the…