About
**Project Description**: CERN maintains file metadata across Rucio, EOS, and the CERN Tape Archive (CTA), making periodic reconciliation necessary to detect inconsistencies such as dark or missing data. This project investigated Apache Airflow as an alternative to Rundeck for orchestrating the reconciliation workflow. A proof-of-concept Airflow deployment was implemented on CERN's Kubernetes infrastructure using Argo CD, Git-synced DAGs, CERN S3, and temporary PostgreSQL storage. Overall, a representative EOS–CTA reconciliation DAG was developed based on the Rundeck implementations. The results show that Airflow can represent the workflow's dependencies, parallel processing, validation, and cleanup stages clearly while providing centralized execution monitoring, although this comes with additional deployment and operational complexity that must be considered before production adoption. **Supervisor**: Idriss Larbi
00:00:00 Slide 1
00:00:21 Slide 2
00:00:42 Slide 3
00:00:56 Slide 4
00:01:19 Slide 5
00:01:37 Slide 6
00:02:09 Slide 7
00:02:26 Slide 8
00:02:54 Slide 9
00:03:22 Slide 10
00:03:53 Slide 11
The talk
Credits and licence
- Institution
- CERN
- Speaker
- Hung Manh Nguyen
- Year
- 2026
- Language
- English
- Source
- CERN
- Licence
- CERN, as CERN states it. CERN's terms
- Attribution
- © CERN. CERN provides its audiovisual media free of charge for educational and informational use, credited to CERN; this lecture is shown in CERN's own player.
- Who can watch
- CERN gives no rating. Invisible Network shows it to Teens profiles and older, judged from what the source says about it. How courses are placed