Data acquisition and on-the-fly processing from high rate detectors at MAX IV

At MAX IV, we have developed a high-performance data acquisition (DAQ) system to handle the high rate detectors exploiting the brightness of the fourth-generation source. This system integrates multiple detector types, including photon counting and charge integrating detectors as well as sCMOS cameras, into a unified DAQ framework. Data are streamed to a central Kubernetes cluster which mounts an IBM Storage Scale (GPFS) storage, with control provided via Tango. The system provides live feedback from the detectors/cameras and is furthermore extended to provide on-the-fly data reduction via the "Dranspose" framework, a horizontally scalable, distributed data analysis pipeline. We present an overview of the diverse detector suite at MAX IV and describe the components of our DAQ and processing framework, highlighting its performance for live data streaming and on-the-fly reduction with reference to several applications.