Operation Optimization Decision-Making of Aluminum Electrolysis Process Using Offline Reinforcement Learning

A common scenario in aluminum electrolysis process is that the collected dataset contains different behavioral policies and some risky policies, such industrial scenario brings new challenges for offline reinforcement learning to learn safety and feasible optimization policy. This article proposes an offline multiobjective reinforcement learning with multicategory policy constraint for operation optimization decision-making (OODM) of the aluminum electrolysis process. The learned optimization policy can surpass the behavior policy while also meet the strict safety requirements of industrial operations. To alleviate the distribution shift problem on multicategory behavioral policy, we present a multicategory policy constraint in the actor network that utilizes the mixture Gaussian variational autoencoder (GMVAE) to implement behavior cloning between the behavioral policy and the learned policy. Based on the actor-critic reinforcement learning architecture, we design two critic networks for multiobjective optimization. Except for the operational performance critic network, an additional safety critic network is introduced to guarantee that the learned policy satisfies the strict safety requirements of industrial operations. We also conduct extensive comparative experiments on the real-world aluminum electrolysis process. Experimental results demonstrate that the proposed method can achieve superior performance against the other offline reinforcement learning algorithms.