Energy management method for virtual power plant based on double-layer deep reinforcement learning game

In order to realize the stable and efficient operation of a Virtual Power Plant (VPP) in a complex power market environment, a VPP energy management method based on a double-layer deep reinforcement learning game is proposed. Firstly, the multi-agent transaction model in VPP is described, and the master-slave game model of combined heat and power operator (CHPO) and electric heating user aggregator (EHUA) in VPP is constructed by designing master-slave object and constraint. Secondly, a two-layer Markov decision process is constructed. Then, aiming at the problem of poor robustness of the traditional optimization method, a master-slave deep decision model based on a soft actor-critic (SAC) algorithm and double-depth Q network (DDQN) algorithm is proposed, and an offline training process of master-slave alternating training is constructed. Finally, simulation results demonstrate that the proposed methodology in this paper achieves optimization of the electricity-heat pricing strategy, estimation of user’s electricity-heat response Q-values, and robust and efficient decision-making in complex environments. This approach successfully accomplishes the synergistic optimization of multi-agent objectives within VPP.