A Robust Reinforcement Learning Control Method for Uncertain Process Industry Based on Knowledge-Constrained Adversarial Perturbation

The process industry is a continuous manufacturing system that comprises intricate physical and chemical reactions. Given the increasing constraints on resources and energy, it is urgent to optimize process indicators by maintaining an efficient reaction atmosphere. Reinforcement learning (RL), using trial and error to learn control strategies, has become a topic of interest in the control community. However, practical implementation reveals that the mapping between observed state variables and the reaction atmosphere is subject to uncertain disturbances, which seriously affect the reliability of process indicator control. To address these issues, a robust RL (RRL) control method based on knowledge-constrained adversarial perturbation is proposed. It applies the adversary to perturb the observed state to characterize the uncertain disturbance. First, the insight of composite modeling for the process industry is presented to factorize the inherent and external uncertainties. Based on this insight, a reaction atmosphere indicator surrogate model is built to quantify the inherent uncertainty. Second, by leveraging the variation boundary information of the surrogate model, a dynamic state perturbation set and its update policy are proposed to ensure the rationality of the state perturbation. Last, an external uncertain time series generation method with continuity constraints is proposed to incorporate reasonable external uncertainty in the training process. Case validation in zinc electrowinning demonstrates that the proposed method effectively enhances control performance in uncertain scenarios.