Learn · CERN

Talk23 min

Analysing I/O bottlenecks in LHC data analysis on grid storage resources

Wahid Bhimji · CERN

Watch At CERN

About

We describe recent I/O testing frameworks that we have developed and applied within the UK GridPP Collaboration, the ATLAS experiment and the DPM team, for a variety of distinct purposes. These include benchmarking vendor supplied storage products, discovering scaling limits of SRM solutions, tuning of storage systems for experiment data analysis, evaluating file access protocols, and exploring IO read patterns of experiment software and their underlying event data models. With multiple grid sites now dealing with petabytes of data, such studies are becoming increasingly essential. We describe how the tests build, and improve, on previous work and contrast how the use-cases differ. We also detail the results obtained and the implications for storage hardware, middleware and experiment software..

0:00:00 Slide 1

0:01:09 Slide 2

0:02:18 Slide 3

0:04:36 Slide 4

0:06:53 Slide 5

0:09:11 Slide 6

0:11:28 Slide 7

0:13:46 Slide 8

0:16:03 Slide 9

0:18:21 Slide 10

0:20:38 Slide 11

The talk

  1. Analysing I/O bottlenecks in LHC data analysis on grid storage resources 2012-05-22 · 23 min · CERN's player

Credits and licence

Institution
CERN
Department
Conferences
Speaker
Wahid Bhimji
Year
2012
Language
English
Subjects
Physics, Computer science, Earth and environment
Source
CERN
Licence
CERN, as CERN states it. CERN's terms
Attribution
© CERN. CERN provides its audiovisual media free of charge for educational and informational use, credited to CERN; this lecture is shown in CERN's own player.
Who can watch
CERN gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed