About
CERN uses the world's largest scientific computing grid, WLCG, for distributed data storage and processing. Monitoring of the CPU and storage resources is an important and essential element to detect operational issues in its systems, for example in the storage elements, and to ensure their proper and efficient function. The processing of experiment data depends strongly on the data access quality, as well as its integrity and both of these key parameters must be assured for the data lifetime. Given the substantial amount of data, O(200PB), already collected by ALICE and kept at various storage elements around the globe, scanning every single data chunk would be a very expensive process, both in terms of computing resources usage and in terms of execution time. In this paper, we describe a distributed file crawler that addresses these natural limits by periodically extracting and analyzing statistically significant samples of files from storage elements, evaluates the results and is integrated with the existing monitoring solution, MonALISA..
0:00:00 Slide 1
0:00:56 Slide 2
0:01:51 Slide 3
0:02:47 Slide 4
0:03:42 Slide 5
0:04:38 Slide 6
0:05:33 Slide 7
0:06:28 Slide 8
0:07:24 Slide 9
0:08:19 Slide 10
0:09:15 Slide 11
The talk
Credits and licence
- Institution
- CERN
- Department
- Conferences
- Speaker
- Adrian-Eduard Negru
- Year
- 2021
- Language
- English
- Subjects
- Physics, Computer science, Earth and environment
- Source
- CERN
- Licence
- CERN, as CERN states it. CERN's terms
- Attribution
- © CERN. CERN provides its audiovisual media free of charge for educational and informational use, credited to CERN; this lecture is shown in CERN's own player.
- Who can watch
- CERN gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed