Real-Time Price-Based Demand Response for Industrial Manufacturing Process via Safe Reinforcement Learning

Industrial manufacturing processes present unique challenges in implementing demand response due to their complex equipment interactions, diverse operation modes, and safety constraints. Conventional model-based optimization methods often struggle in this context, requiring complete mathematical models and accurate uncertainty distributions. In this article, we propose a model-free safe deep reinforcement learning approach for real-time scheduling of industrial manufacturing systems, ensuring constraint adherence and accommodating diverse equipment operation modes. Specifically, the approach formulates the industrial demand response problem as a constrained Markov decision process with hybrid action space, capturing the intricate interplay between variable-speed equipment and discrete actuators, while considering safety constraints. To enhance exploration and robustness, the proposed method combines Lagrange multipliers based on soft actor–critic to satisfy the constraints. The cross-attention mechanism is utilized to find the association rules between hybrid actions to solve the challenges posed by the spatial gradient of hybrid actions. The proposed approach is trained and tested on a real-world dataset, demonstrating its superior performance in achieving significant cost reductions for manufacturing while satisfying operational constraints. Furthermore, sensitivity analysis underpins robustness against the variable real-time prices, showcasing its industrial applicability.