A Two-layer Optimal Planning Method for Integrated Energy Systems based on Reinforcement Learning

With the continuous development of new energy technologies, regional integrated energy system (RIES) is seen as an energy utilization model with great potential. Given that the RIES encompasses various forms of energy subsystems, the model exhibits high dimensionality, posing significant challenges for joint planning and scheduling. Based on this, this paper proposes a two-layer planning optimization method based on reinforcement learning. The upper layer model using deep deterministic policy gradient (DDPG) algorithm to find the optimal capacity configuration by continuously interacting with the environment. The lower layer model uses the annual electricity, heat, and cooling loads and the equipment capacity solved by the upper layer as inputs for optimal scheduling of typical days. The scheduling cost calculated by the lower layer is fed back to the upper layer as part of the environmental return, thereby optimizing the capacity configuration. The case test results verify the effectiveness of the proposed model and method.