About
From 1VM+LUN to k8s+S3 - an uneasy way…
Since 2015 PSNC has provided a sync & share service for science and academia in Poland, based on the Seafile software. We started small by running a 1VM+1LUN setup and community version of the software, integrated with the local PSNC's LDAP. In 2016 we began to build a fully-fledged setup based on a cluster of application servers, background jobs servers, DB servers, and a dedicated 2-servers GPFS cluster as a storage backend, that become operational in 2017 and serves most of our users until today. We are currently migrating our service towards the most modern and fancy setup based on k8s and Ceph/S3.
In our presentation, we discuss the experiences and observations on the impact that changing cloud and storage technologies and infrastructure compute/storage infrastructures' features have on your system, services, data, and users, while you are trying to follow the trends in the system architectures, services deployment approaches, management practices, etc. We also discuss the pros and cons of the simplified 1-VM setup vs the bare-metal multiserver infrastructure vs the fully containerized setup with lots of automation.
Surprisingly (or not) we faced the unobvious and uneasy-to-accept fact, that the 'ancient' simplistic setup of our sync & share system required the least effort to keep it up, and caused almost no operational issues, faults & failures over 8+ years of operation, while the complexity of the management processes such as data and users migration, systems and application upgrades grows non-linearly with growing the level of 'fanciness' and 'intelligence' of the infrastructure and application setup. Obviously, it would not be fair to say that the capabilities of a 1VM+1LUN platform intended to serve ~500 users with <40TB data and ~8 million files are comparable to a fully-fledged clustered setup for 1000s of users with ~1PB storage and quarter billion files.
Therefore we will explain the reasoning behind the…
The talk
Credits and licence
- Institution
- CERN
- Department
- HEP Computing
- Speaker
- Krzysztof Wadówka
- Year
- 2023
- Language
- English
- Subjects
- Computer science
- Source
- CERN
- Licence
- CERN, as CERN states it. CERN's terms
- Attribution
- © CERN. CERN provides its audiovisual media free of charge for educational and informational use, credited to CERN; this lecture is shown in CERN's own player.
- Who can watch
- CERN gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed