Reinforcement Learning-based Constant Flux Control of Membrane Filtration Process

Submerged membrane bioreactor (SMBR) systems offer high treatment efficiency and compact operation, but nonlinear dynamics and time-varying membrane fouling make constant-flux regulation challenging. This study develops and evaluates three reinforcement-learning controllers—a deep Q-network (DQN), deep deterministic policy gradient (DDPG), and a proposed adaptive constrained DDPG (AC-DDPG)—using a recurrent-neural-network model identified from a laboratory SMBR pilot plant as the interactive environment. AC-DDPG augments conventional DDPG with bounded tracking-error-dependent exploration and a dual-timescale critic architecture, enabling the controller to emphasize rapid transient regulation during large deviations while improving long-horizon accuracy near the setpoint. The state, action, reward, safety constraints, and training protocol are explicitly formulated for reproducibility. Across five paired training trials, AC-DDPG reduced the mean convergence episode from \\(75.6\\pm 2.6\\) to \\(27.4\\pm 4.6\\) , with a paired mean reduction of 48.2 episodes (95% confidence interval: 42.6–53.8). The reported average tracking error decreased from 0.853 to 0.468, corresponding to a 45.1% reduction. Response-curve analysis further shows that AC-DDPG achieves a favorable balance of fast settling, low overshoot, reduced steady-state error, and effective disturbance recovery compared with the evaluated DQN and DDPG controllers. These results demonstrate the effectiveness of the proposed adaptive exploration and dual-timescale learning framework for data-driven continuous flux control and provide a technically grounded basis for subsequent pilot-plant implementation.