Using S3 storage with XRootD vs Dynafed

The High Energy Physics Research Computing group at UVic used Dynafed to create federated storage clusters for Belle II and ATLAS that were used by worker nodes deployed on cloud computing facilities around the world. Since the development on Dynafed stopped, XRootD was tested with S3 as backend to replace Dynafed. We will show the similarities as well as the major differences between the two systems for data transfers, checksum calculations as well as clustering of different endpoints, which may help others to efficiently make use of S3 storage as a WLCG site SE.

Authors

Institutions

Publication Details

Journal
EPJ Web of Conferences
Published
2025-01-01
DOI
https://doi.org/10.1051/epjconf/202533701053
Primary Topic
Distributed and Parallel Computing Systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Using S3 storage with XRootD vs Dynafed

Colson Driemel, R. Sobie, M. Ebert, T. Sullivan
EPJ Web of Conferences
Distributed and Parallel Computing Systems
article

Using S3 storage with XRootD vs Dynafed

Colson Driemel, R. Sobie, M. Ebert, T. Sullivan
article en

Abstract

The High Energy Physics Research Computing group at UVic used Dynafed to create federated storage clusters for Belle II and ATLAS that were used by worker nodes deployed on cloud computing facilities around the world. Since the development on Dynafed stopped, XRootD was tested with S3 as backend to replace Dynafed. We will show the similarities as well as the major differences between the two systems for data transfers, checksum calculations as well as clustering of different endpoints, which may help others to efficiently make use of S3 storage as a WLCG site SE.

EPJ Web of ConferencesVol. 337
University of Victoria (CA), Institute of Particle Physics (CA)
Openalex Percentile: Top 68%
Distributed and Parallel Computing Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.