Evolving StoRM WebDAV: Delegation of file transfers to NGINX and support for SciTags

After the deprecation of the open-source Globus Toolkit used for GridFTP transfers, the WLCG community has shifted its focus to the HTTP protocol. The WebDAV protocol extends HTTP to create, move, copy and delete resources on web servers. StoRM WebDAV provides data storage access and management through the WebDAV protocol over a POSIX file system. Mainly designed to be used by the WLCG community, StoRM WebDAV supports authentication through X.509 certificates, VOMS proxies and JSON Web Tokens (JWT). Moreover, Third-Party Copies (an extension of the WebDAV COPY verb to support copies between data centers) are supported. With the aim of improving data transfer performance, this contribution describes the changes made to StoRM WebDAV in order to delegate file transfers to the external reverse proxy NGINX, decoupling them from the internal Java implementation. To simplify even more the StoRM WebDAV codebase, also the validation of VOMS proxies and JWTs is delegated to NGINX, augmented with specific modules developed by us. Even with this solution, authorization is still enforced by StoRM WebDAV. Following the effort of the WLCG community to have better metrics about data flows, this contribution also describes the work done in order to support SciTags, an initiative promoting identification of the science domains and their high-level activities at the network level.

Authors

Institutions

Publication Details

Journal
EPJ Web of Conferences
Published
2025-01-01
DOI
https://doi.org/10.1051/epjconf/202533701182
Primary Topic
Scientific Computing and Data Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Evolving StoRM WebDAV: Delegation of file transfers to NGINX and support for SciTags

Enrico Vianello, Luca Bassi, Jacopo Gasparetto, F. Giacomini et al.
EPJ Web of Conferences
Scientific Computing and Data Management
article

Evolving StoRM WebDAV: Delegation of file transfers to NGINX and support for SciTags

Enrico Vianello, Luca Bassi, Jacopo Gasparetto, F. Giacomini, Federica Agostini, Roberta Miccoli
article en

Abstract

After the deprecation of the open-source Globus Toolkit used for GridFTP transfers, the WLCG community has shifted its focus to the HTTP protocol. The WebDAV protocol extends HTTP to create, move, copy and delete resources on web servers. StoRM WebDAV provides data storage access and management through the WebDAV protocol over a POSIX file system. Mainly designed to be used by the WLCG community, StoRM WebDAV supports authentication through X.509 certificates, VOMS proxies and JSON Web Tokens (JWT). Moreover, Third-Party Copies (an extension of the WebDAV COPY verb to support copies between data centers) are supported. With the aim of improving data transfer performance, this contribution describes the changes made to StoRM WebDAV in order to delegate file transfers to the external reverse proxy NGINX, decoupling them from the internal Java implementation. To simplify even more the StoRM WebDAV codebase, also the validation of VOMS proxies and JWTs is delegated to NGINX, augmented with specific modules developed by us. Even with this solution, authorization is still enforced by StoRM WebDAV. Following the effort of the WLCG community to have better metrics about data flows, this contribution also describes the work done in order to support SciTags, an initiative promoting identification of the science domains and their high-level activities at the network level.

EPJ Web of ConferencesVol. 337
Consortium GARR (IT), Istituto Nazionale di Fisica Nucleare, Sezione di Bologna (IT)
Openalex Percentile: Top 73%
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.