A dual-reward dual optimization model based on hybrid RL for dynamic optimization of steel energy systems

In steel energy systems, an optimal assignment of multiple energy media of gas, electricity and steam is crucial. Reinforcement learning (RL) becomes an effective approach to realize dynamic strategy allocation. In the system modeling process, on the one hand, it is necessary to learn the varying characteristics of energy production and consumption from data, on the other hand, it relies on prediction for electricity load or energy storage, which makes the learning environment contain both model-free and model-based state transfer process. To solve this problem, a dual-reward dual RL method is proposed, which makes the dual part dynamically adjustable according to the prediction results and constraint violation degree by introducing an adaptive dual variable update strategy. Furthermore, to address the problem of uncertainty in the model-based part, the deviation of the model-based part from the real data was quantified by constructing different confidence intervals to alleviate the impact of uncertainty on the strategy. In order to verify the effectiveness of the proposed method, experiments are conducted using real data from the energy system of a domestic steel plant and compared with the traditional RL method.