Deep reinforcement learning for integrated data assimilation and valve control in water distribution systems

• Developed a DRL framework for integrated prediction and automated control. • Improved water-quality prediction accuracy over benchmark methods. • Captured time-varying reaction coefficients through dynamic calibration. • Effectively identified, isolated, and flushed contamination incidents. • Enhanced both modeling performance and response efficiency in WDSs. Drinking water is delivered to end users through complex water-distribution systems (WDSs), which are typical of nearly all urbanized areas. Contaminants can persist and spread within these networks due to insufficient sterilization at treatment facilities or external contamination. Accurate prediction of contaminant propagation and the development of automated mitigation strategies are therefore critical. This study presents deep reinforcement learning (DRL)-based models to enhance water-quality management in WDSs. The first DRL model, which integrates Environmental Protection Agency Network Evaluation Tool 2.2 (EPANET) simulations with real-time sensor data to optimize reaction coefficients, outperformed both benchmark methods across all six sensors during the validation period. It achieved average R² improvements of 20.9% and 21.2% and RMSE reductions of 11.3% and 21.7% relative to the ensemble Kalman filter and pattern-search method, respectively. Performance gains were most pronounced at terminal nodes, where water stagnates and reaction coefficients have a stronger influence than in high-circulation zones. The second DRL model employs a double deep Q-network to automatically operate valves and hydrants upon contaminant detection, achieving an average 90.7% contaminant removal within 5 h across 100 simulated incidents. Overall, the automated management system effectively identified and isolated contaminated areas, highlighting the potential of a DRL-based framework to improve both predictive modeling and operational responsiveness, and underscoring the importance of data assimilation and autonomous control in modern water-quality management.