Safe reinforcement learning for wastewater treatment with an input convex safety critic

Wastewater treatment plays a crucial role in resource recovery and sustainability. Developing a control strategy that reduces energy consumption while maintaining effluent quality remains a challenge because the treatment process is characterised by its highly energy-intensive nonlinear dynamics. To tackle this challenge, this study proposes a novel hierarchical control strategy that integrates deep reinforcement learning with an input-convex safety critic. In the upper layer, the agent determines the optimal dissolved oxygen setpoints and internal recirculation flow rates, while in the lower layer, conventional feedback control regulates aeration through the tracking of dissolved oxygen setpoints. The performance of the proposed method is compared with constant DO control, ammonia-based aeration control, model predictive control and reinforcement learning methods using the Benchmark Simulation Model 1 under different operating conditions. The results demonstrate that the proposed method achieves up to 14.0% reduction in total energy consumption compared to constant DO control. Additionally, it significantly decreases the effluent quality violation duration in terms of total nitrogen and ammonia nitrogen by up to 72.0% and 60.5%, respectively, demonstrating its capability to balance energy efficiency and safety requirements in wastewater treatment.