Label-based Virtual Directories In dCache

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

Authors

Institutions

Publication Details

Journal
EPJ Web of Conferences
Published
2025-01-01
DOI
https://doi.org/10.1051/epjconf/202533701274
Primary Topic
Multimedia Communication and Technology
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Label-based Virtual Directories In dCache

Lea Morschel, Marina Sahakyan, Svenja Meyer, Krishnaveni Chitrapu et al.
EPJ Web of Conferences
Multimedia Communication and Technology
article

Label-based Virtual Directories In dCache

Lea Morschel, Marina Sahakyan, Svenja Meyer, Krishnaveni Chitrapu, Christopher Green, Tigran Mkrtchyan, Dmitry Litvintsev
article en

Abstract

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

EPJ Web of ConferencesVol. 337
Fermi National Accelerator Laboratory (US), Deutsches Elektronen-Synchrotron DESY (DE), National Supercomputing Center of Tianjin (CN)
Openalex Percentile: Top 63%
Multimedia Communication and Technology
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Label-based Virtual Directories In dCache — Lea Morschel, Marina Sahakyan, et al. · EPJ Web of Conferences (2025) | TGRS Research Map | TGRS