Data automation strategy at ESRF

The ESRF has initiated an ambitious program to automate data processing and management across all 45 of its beamlines. Since the successful installation and commissioning of the ESRF-EBS new storage ring in 2020, there has been a significant increase in X-ray flux. A comprehensive strategy has been developed to optimize the use of the increasing quantities of data generated by the new advanced detectors. This strategy consists of several components, including the Lima2 project [1], which provides scalable detector control and early data processing and reduction. The new beamline control system, BLISS [2], coordinates synchronization and data acquisition. Additionally, the Data Automation software, EWOKS [3], processes and exploits the produced data by running data processing workflows either triggered by BLISS online or run offline. DRAC [4] complements the strategy by implementing a database and associated tools for managing all data and metadata generated during experiments and the associated automated pipelines. We will examine how DRAC, EWOKS, and BLISS work together, the techniques that have already been automated, the potential extent of data automation and the orchestration function of Blissdata. [1] Lima2: ESRF Software Strategy for high-throughput detectors (this conference) [2] https://doi.org/10.1080/08940886.2023.2277141 [3] https://doi.org/10.1080/08940886.2024.2432305 [4] https://doi.org/10.1080/08940886.2024.2432816