Keep-up Production in JUNO’s Offline Data Processing
The Jiangmen Underground Neutrino Observatory (JUNO) is a cutting-edge scientific experiment designed to address fundamental questions in neutrino physics, including the determination of the neutrino mass ordering and precision measurements of neutrino oscillation parameters. To support the massive data processing requirements of JUNO, we have developed the Keep Up Production (KUP) pipeline, a robust and scalable system for managing the offline data processing workflow. This paper presents the architecture, design, and implementation of the KUP pipeline, highlighting its key components, including job management, data visualization, and the use of modern web technologies. We also discuss the challenges posed by the complexity and volume of data, and how the KUP pipeline addresses these challenges through automation, modularity, and real-time monitoring.
Authors
- Yizhou Zhang (ORCID: https://orcid.org/0000-0002-8206-4694)
- Tao Lin (ORCID: https://orcid.org/0000-0002-6450-9629)
- Weiqing Yin
Institutions
- Chinese Academy of Sciences (CN)
- University of Chinese Academy of Sciences (CN)
Publication Details
- Journal
- EPJ Web of Conferences
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1051/epjconf/202533701176
- Primary Topic
- Distributed and Parallel Computing Systems
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Natural Science Foundation of China
- Chinese Academy of Sciences
- Youth Innovation Promotion Association