Human-in-the-loop reinforcement learning for data quality monitoring in particle physics experiments

Abstract Ensuring data integrity via data quality monitoring (DQM) is critical in large-scale particle physics experiments. This traditionally relies on labor-intensive manual inspection or static machine learning models, which are adequate for stable operating conditions. However, the current era of major detector upgrades at the large hadron collider (LHC) challenges this paradigm. These upgrades are followed by prolonged commissioning periods with frequent and unpredictable changes in detector conditions, a regime for which static models are ill-suited. To address this, we reframe DQM as a dynamic decision-making problem and introduce a human-in-the-loop reinforcement learning (RLHL) framework. Our proximal policy optimization agent learns both to classify data and to strategically decide when to query human experts, optimizing the automation-oversight balance. On synthetic data, the system rapidly adapts to abrupt condition changes and successfully learns from noisy labels. In a simulated online regime, the agent minimizes human intervention by requesting it mainly when its uncertainty is high. A preliminary study on a real offline dataset from the LHCb experiment demonstrates that our synthetic approach is a reasonable proxy for real-world scenarios. The algorithm generalizes effectively with only superficial hyperparameter tuning, robustly identifying anomalies even when trained on augmented data. This work presents a scalable, adaptive solution for semi-autonomous DQM.