Current CERN platforms for Reproducible and Interactive scientific analysis

About

Experiment analysis frameworks, physics data formats, and the expectations of LHC scientists have evolved towards including interactive analysis with short turnaround times and the possibility to optimize reproducible and re-interpretable workflows. The CERN IT's Pilot Analysis Facility, the CERN Virtual Research Environment, and REANA have emerged as key solutions, as well as a platform dedicated to Machine Learning. REANA is an open source software, developed mostly within the CERN IT department, that provides a framework focussed on running containerised analysis workflows with an emphasis on the re-analysis and reproducibility of scientific results. We shall describe how REANA has been used notably by the ATLAS collaboration for pMSSM reinterpretations of LHC Run 2 analyses. The Pilot Analysis Facility facilitates interactive, notebook-based analysis and enables scaling out from SWAN to the local HTCondor managed extensive compute resources. Similarly, the Virtual Research Environment offers an inclusive, CERN-independent authentication mechanism allowing registered users to benefit from an interactive analysis environment that can be interfaced directly with data management (such as Rucio) and reproducibility (e.g. REANA) tools. These tools form an integrated ecosystem enabling users to conduct, manage, and replicate complex analyses while promoting accessibility and collaboration. By emphasising user-friendly interfaces and middleware, it optimises data analysis from access and computations to replicable scientific outcomes, fostering open collaboration across diverse physics communities. This is complemented by the CERN ML Project ( ml.cern.ch ) that aims at offering a centralized service to manage the full machine learning lifecycle, providing access to various accelerator resources like GPUs, TPUs and FPGAs..

0:00:00 Slide 1

0:02:29 Slide 2

0:04:41 Slide 3

0:05:46 Slide 4

0:10:31 Slide 5

0:11:55 Slide 6

0:14:27 Slide 7