Benchmarking the Open Science Data Federation services to develop XRootD best practices

Research has become dependent on processing power and storage, one crucial aspect being data sharing. The Open Science Data Federation (OSDF) project aims to create a scientific global data distribution network based on the Pelican Platform. OSDF relies on the XRootD and Pelican projects. Nevertheless, OSDF must understand the XRootD limits under various configuration options, including transfer rate limits, proper buffer configuration, and storage type effect. We have thus executed a set of benchmarks to create a set of recommendations to share with the XRootD and Pelican teams. This work describes the tests and results performed using National Research Platform (NRP) hosts. The tests cover various file sizes and parallel streams and use clients from various distances from the server host. We also used several standalone clients (wget, curl, pelican) and the native HTCondor file transfer mechanisms. Applying the methodology creates a possibility to track how XRootD and the Pelican layer perform in different scenarios.

Authors

Publication Details

Journal
Springer Link (Chiba Institute of Technology)
Published
2025-10-07
DOI
https://doi.org/10.1051/epjconf/202533701256/pdf
Primary Topic
Scientific Computing and Data Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Benchmarking the Open Science Data Federation services to develop XRootD best practices

Fábio Andrijauskas, Igor Sfiligoi, Frank Würthwein
Springer Link (Chiba Institute of Technology)
Scientific Computing and Data Management
article

Benchmarking the Open Science Data Federation services to develop XRootD best practices

Fábio Andrijauskas, Igor Sfiligoi, Frank Würthwein
article en

Abstract

Research has become dependent on processing power and storage, one crucial aspect being data sharing. The Open Science Data Federation (OSDF) project aims to create a scientific global data distribution network based on the Pelican Platform. OSDF relies on the XRootD and Pelican projects. Nevertheless, OSDF must understand the XRootD limits under various configuration options, including transfer rate limits, proper buffer configuration, and storage type effect. We have thus executed a set of benchmarks to create a set of recommendations to share with the XRootD and Pelican teams. This work describes the tests and results performed using National Research Platform (NRP) hosts. The tests cover various file sizes and parallel streams and use clients from various distances from the server host. We also used several standalone clients (wget, curl, pelican) and the native HTCondor file transfer mechanisms. Applying the methodology creates a possibility to track how XRootD and the Pelican layer perform in different scenarios.

Springer Link (Chiba Institute of Technology)
Industry, innovation and infrastructure, Partnerships for the goals
Openalex Percentile: Top 11%
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Benchmarking the Open Science Data Federation services to develop XRootD best practices — Fábio Andrijauskas, Igor Sfiligoi, et al. · Springer Link (Chiba Institute of Technology) (2025) | TGRS Research Map | TGRS