Incremental Critic Learning Control With Policy Transfer for Wastewater Treatment Processes

Regarding the control of dissolved oxygen concentration in wastewater treatment plants (WWTPs), the direct control policies in conventional critic learning control algorithms often exhibit high sensitivity to system disturbances, which may potentially causing significant degradation in control performance. To address this limitation, a transferable incremental critic learning control (TICLC) algorithm is developed, reformulating the control policy into an incremental form. Furthermore, the historical operation data of WWTPs is leveraged to enable policy transfer across different control methods, which effectively improves the control performance of the TICLC algorithm during the initial learning phase. In addition, a novel utility function incorporating a soft policy constraint mechanism is designed to enhance the stability of the TICLC algorithm during the online optimization. The stability of the TICLC algorithm is proved by constructing a Lyapunov function. Experimental results derived from the benchmark simulation model no. 1 show that the TICLC algorithm achieves superior control performance for the dissolved oxygen concentration in WWTPs.