Data management infrastructure for European XFEL

Effective data management is crucial for accessible and usable research data. This presentation will describe the data infrastructure at the European XFEL, built on a four-layer storage architecture designed for high-throughput handling. The first layer, Online storage, acts as a high-speed cache for data rates up to 15 GB/s. The second layer, High-Performance Storage, supports real-time processing and analysis, both connected via a 1 Tb/s InfiniBand link spanning 4.4 km to the DESY computing center. The third layer, Mass Storage, enables mid-term access for in-depth analysis, while the Tape Archive ensures secure long-term storage beyond 10 years. Integrated with computing clusters, the system supports both near-online and offline analysis, or data export. Capable of processing up to 2 PB of data per day, the infrastructure demonstrates exceptional performance and reliability.