dCache CI/CD migration to Kubernetes

For over two decades, the dCache project has provided open-source to satisfy ever-more demanding storage requirements. More than 80 sites worldwide rely on dCache to provide services for LHC experiments, Belle-II, Eu- XFEL, and others. This can be achieved only with a well-established process from a whiteboard, where ideas are created through development, packaging, and testing. The project’s build and test infrastructure is based on Jenkins CI and a set of virtual machines. This infrastructure is maintained by dCache developers. With the introduction of the DESY-central Gitlab server, the developers have started migrating from VM-based testing to container-based deployments in the onsite Kubernetes cluster. As a result, we have packaged dCache containers and Helm charts that can be used by other sites to reproduce our test and build steps quickly or to evaluate new releases on their pre-production systems and, eventually, become a standard model of dCache deployment at the sites. This paper describes the challenges we have faced, the techniques we used to solve them, and the issues that still need to be addressed.

Authors

Publication Details

Journal
Springer Link (Chiba Institute of Technology)
Published
2025-10-07
DOI
https://doi.org/10.1051/epjconf/202533701103/pdf
Primary Topic
Distributed and Parallel Computing Systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

dCache CI/CD migration to Kubernetes

Markus Wengert, Stefan Bujack, Tobias Klann, Lea Morschel et al.
Springer Link (Chiba Institute of Technology)
Distributed and Parallel Computing Systems
article

dCache CI/CD migration to Kubernetes

Markus Wengert, Stefan Bujack, Tobias Klann, Lea Morschel, Dmitry Litvintsev, Michael Schuh, Tigran Mkrtchyan, Marina Sahakyan, Svenja Meyer, Christopher Green
article en

Abstract

For over two decades, the dCache project has provided open-source to satisfy ever-more demanding storage requirements. More than 80 sites worldwide rely on dCache to provide services for LHC experiments, Belle-II, Eu- XFEL, and others. This can be achieved only with a well-established process from a whiteboard, where ideas are created through development, packaging, and testing. The project’s build and test infrastructure is based on Jenkins CI and a set of virtual machines. This infrastructure is maintained by dCache developers. With the introduction of the DESY-central Gitlab server, the developers have started migrating from VM-based testing to container-based deployments in the onsite Kubernetes cluster. As a result, we have packaged dCache containers and Helm charts that can be used by other sites to reproduce our test and build steps quickly or to evaluate new releases on their pre-production systems and, eventually, become a standard model of dCache deployment at the sites. This paper describes the challenges we have faced, the techniques we used to solve them, and the issues that still need to be addressed.

Springer Link (Chiba Institute of Technology)
Industry, innovation and infrastructure
Openalex Percentile: Top 27%
Distributed and Parallel Computing Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.