Using S3 storage with XRootD vs Dynafed

The High Energy Physics Research Computing group at UVic used Dynafed to create federated storage clusters for Belle II and ATLAS that were used by worker nodes deployed on cloud computing facilities around the world. Since the development on Dynafed stopped, XRootD was tested with S3 as backend to replace Dynafed. We will show the similarities as well as the major differences between the two systems for data transfers, checksum calculations as well as clustering of different endpoints, which may help others to efficiently make use of S3 storage as a WLCG site SE.

Authors

Publication Details

Journal
Springer Link (Chiba Institute of Technology)
Published
2025-10-07
DOI
https://doi.org/10.1051/epjconf/202533701053/pdf
Primary Topic
Distributed and Parallel Computing Systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Using S3 storage with XRootD vs Dynafed

Colson Driemel, Marcus Ebert, Randall Sobie, Tristan Sullivan
Springer Link (Chiba Institute of Technology)
Distributed and Parallel Computing Systems
article

Using S3 storage with XRootD vs Dynafed

Colson Driemel, Marcus Ebert, Randall Sobie, Tristan Sullivan
article en

Abstract

The High Energy Physics Research Computing group at UVic used Dynafed to create federated storage clusters for Belle II and ATLAS that were used by worker nodes deployed on cloud computing facilities around the world. Since the development on Dynafed stopped, XRootD was tested with S3 as backend to replace Dynafed. We will show the similarities as well as the major differences between the two systems for data transfers, checksum calculations as well as clustering of different endpoints, which may help others to efficiently make use of S3 storage as a WLCG site SE.

Springer Link (Chiba Institute of Technology)
Affordable and clean energy
Openalex Percentile: Top 27%
Distributed and Parallel Computing Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.