On-the-fly data set combinations with RNTuple

With the expected data volume increase for HL-LHC and the even more complex computing challenges set by future colliders, the need for efficient data storage and processing becomes more pressing. ROOT’s next-generation data format and I/O subsystem, RNTT<ns0:sc>uple</ns0:sc>, is designed to address these challenges. RNTT<ns0:sc>uple</ns0:sc> already demonstrates a clear improvement in storage and I/O efficiency, as well as overall stability and robustness with respect to its predecessor, TTT<ns0:sc>ree</ns0:sc>. These improvements provide a solid baseline to introduce novel extensions to common high-energy and nuclear physics (HENP) workflows. Notably, many workflows could benefit from the ability to arbitrarily join and chain data set samples at runtime, which could reduce overall storage requirements and improve application runtime and ergonomics. In this paper, we present the RNT<ns0:sc>uple</ns0:sc>P<ns0:sc>rocessor</ns0:sc>, which enables HENP data set combinations with RNT<ns0:sc>uple</ns0:sc>. We will discuss the main design considerations, present the interfaces to support data set combinations and show how they integrate in typical workflows.

Authors

Publication Details

Journal
Springer Link (Chiba Institute of Technology)
Published
2025-10-07
DOI
https://doi.org/10.1051/epjconf/202533701013/pdf
Primary Topic
Distributed and Parallel Computing Systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

On-the-fly data set combinations with RNTuple

Philippe Canal, Vincenzo Eduardo Padulano, Ana-Lucia Varbanescu, Jakob Blomer et al.
Springer Link (Chiba Institute of Technology)
Distributed and Parallel Computing Systems
article

On-the-fly data set combinations with RNTuple

Philippe Canal, Vincenzo Eduardo Padulano, Ana-Lucia Varbanescu, Jakob Blomer, Florine Willemijn de Geus
article en

Abstract

With the expected data volume increase for HL-LHC and the even more complex computing challenges set by future colliders, the need for efficient data storage and processing becomes more pressing. ROOT’s next-generation data format and I/O subsystem, RNTT<ns0:sc>uple</ns0:sc>, is designed to address these challenges. RNTT<ns0:sc>uple</ns0:sc> already demonstrates a clear improvement in storage and I/O efficiency, as well as overall stability and robustness with respect to its predecessor, TTT<ns0:sc>ree</ns0:sc>. These improvements provide a solid baseline to introduce novel extensions to common high-energy and nuclear physics (HENP) workflows. Notably, many workflows could benefit from the ability to arbitrarily join and chain data set samples at runtime, which could reduce overall storage requirements and improve application runtime and ergonomics. In this paper, we present the RNT<ns0:sc>uple</ns0:sc>P<ns0:sc>rocessor</ns0:sc>, which enables HENP data set combinations with RNT<ns0:sc>uple</ns0:sc>. We will discuss the main design considerations, present the interfaces to support data set combinations and show how they integrate in typical workflows.

Springer Link (Chiba Institute of Technology)
Affordable and clean energy
Openalex Percentile: Top 27%
Distributed and Parallel Computing Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.