ATLAS HL-LHC Demonstrators with Data Carousel: Dataon-Demand and Tape Smart Writing
The High Luminosity upgrade to the LHC (HL-LHC) is expected to deliver scientific data at the multi-exabyte scale. To tackle this unprecedented data storage challenge, the ATLAS experiment initiated the Data Carousel project in 2018. Data Carousel is a tape-driven workflow in which bulk production campaigns with input data resident on tape are executed by staging and promptly processing a sliding window to disk buffer such that only a small fraction of inputs are pinned on disk at any one time. Put in ATLAS production before Run3, Data Carousel continues to be our focus for seeking new opportunities in disk space savings, and enhancing tape usage throughout the ATLAS Distributed Computing (ADC) environment. These efforts are highlighted by two recent ATLAS HL-LHC demonstrator projects: data-on-demand and tape smart writing. In this paper, we will discuss the recent studies and outcomes from these projects. The research was conducted together with site experts at CERN and Tier-1 centers.
Authors
- Tatiana Korchuganova (ORCID: https://orcid.org/0000-0001-5792-8182)
- Julien Leduc
- Doris Ressmann
- M. Lassnig (ORCID: https://orcid.org/0000-0002-9541-0592)
- A. Klimentov (ORCID: https://orcid.org/0000-0003-2748-4829)
- Xin Zhao (ORCID: https://orcid.org/0000-0001-9121-6090)
- Tadashi Maeno
- M. Borodin
- H. Musheghyan
Institutions
- Karlsruhe Institute of Technology (DE)
- University of Iowa (US)
- University of Pittsburgh (US)
- Brookhaven National Laboratory (US)
- European Organization for Nuclear Research (CH)
Publication Details
- Journal
- EPJ Web of Conferences
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1051/epjconf/202533701350
- Primary Topic
- Distributed and Parallel Computing Systems
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- U.S. Department of Energy
- Office of Science
- High Energy Physics
- Brookhaven National Laboratory