A Pilot Analysis Facility at CERN: Architecture, Implementation and First Evaluation

Experiment analysis frameworks, physics data formats and expectations of scientists at the LHC have been evolving towards interactive analysis with short turnaround times. In preparation for HL-LHC the experiments are moving to data formats suitable for columnar analysis such as RNTuple. In addition, the Python ecosystem is becoming increasingly popular for analysis. Several sites in the community have reacted by setting up dedicated Analysis Facilities, providing tools and interfaces to computing and storage resources suitable for interactive analysis. It is expected that the demand for such facilities will increase towards the HL-LHC era and scaling out interactive processing of large datasets could become necessary. CERN IT launched a Pilot of an Analysis Facility based on established, proven services such as SWAN, HTCondor and EOS. This facilitates the access to massive resources by enabling the use of HTCondor managed resources from SWAN, offering parallel execution via analysis interfaces, such as ROOT RDataFrame and coffea and their Dask back-ends. In this contribution we will discuss the architecture of the Pilot Analysis Facility at CERN, giving the rationale for the decisions. For determining the next steps, evaluating the impact of different resource allocation strategies at the CERN HTCondor pool is critical. One especially interesting strategy consists in combining a set of dedicated resources for interactive analysis with the use of the general resources that are subject to experiment quotas. We will put a special focus on the feedback we received from the early testers from the experiments.

Authors

Publication Details

Journal
Springer Link (Chiba Institute of Technology)
Published
2025-10-07
DOI
https://doi.org/10.1051/epjconf/202533701360/pdf
Primary Topic
Computational Physics and Python Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Pilot Analysis Facility at CERN: Architecture, Implementation and First Evaluation

Markus W. Schulz, Enrique Garcia Garcia, Andrea Sciabà, Ben Jones et al.
Springer Link (Chiba Institute of Technology)
Computational Physics and Python Applications
article

A Pilot Analysis Facility at CERN: Architecture, Implementation and First Evaluation

Markus W. Schulz, Enrique Garcia Garcia, Andrea Sciabà, Ben Jones, Giovanni Guerrieri, Antonio Delgado Peris, Enric Tejedor Saavedra, Diogo Castro
article en

Abstract

Experiment analysis frameworks, physics data formats and expectations of scientists at the LHC have been evolving towards interactive analysis with short turnaround times. In preparation for HL-LHC the experiments are moving to data formats suitable for columnar analysis such as RNTuple. In addition, the Python ecosystem is becoming increasingly popular for analysis. Several sites in the community have reacted by setting up dedicated Analysis Facilities, providing tools and interfaces to computing and storage resources suitable for interactive analysis. It is expected that the demand for such facilities will increase towards the HL-LHC era and scaling out interactive processing of large datasets could become necessary. CERN IT launched a Pilot of an Analysis Facility based on established, proven services such as SWAN, HTCondor and EOS. This facilitates the access to massive resources by enabling the use of HTCondor managed resources from SWAN, offering parallel execution via analysis interfaces, such as ROOT RDataFrame and coffea and their Dask back-ends. In this contribution we will discuss the architecture of the Pilot Analysis Facility at CERN, giving the rationale for the decisions. For determining the next steps, evaluating the impact of different resource allocation strategies at the CERN HTCondor pool is critical. One especially interesting strategy consists in combining a set of dedicated resources for interactive analysis with the use of the general resources that are subject to experiment quotas. We will put a special focus on the feedback we received from the early testers from the experiments.

Springer Link (Chiba Institute of Technology)
Openalex Percentile: Top 15%
Computational Physics and Python Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.