Multiagent Primal-Dual DDPG-Based Reactive Power Optimization of Active Distribution Networks via Graph Reinforcement Learning

The large-scale integration of distributed energy resources into active distribution networks may significantly intensify voltage fluctuations and increase network losses. Traditional model-based reactive power optimization approaches depend on existence of accurate system models. On the other hand, conventional reinforcement learning methods largely ignore the spatial characteristics of the active distribution networks during training, allowing agents to have an inadequate perception of the system state. To address these challenges, this paper proposes a multi-agent deep reinforcement learning approach that integrates graph learning with reinforcement learning for the learning of reactive power optimization strategies in active distribution networks. Specifically, the active distribution network is divided into multiple regions, with each region being controlled by an agent. The agents collaborate to achieve the global reactive power optimization goal. The perception capability of the agents is enhanced by adopting graph attention networks during the feature extraction phase. In the training phase, a primal-dual method is employed to manage constraints effectively. During the execution phase, each agent controls the photovoltaic inverters, electric springs, and capacitor banks based on the strategies developed in the training phase. The performance of the proposed approach is validated by a series of experiments on the IEEE-33 system, along with comparisons versus some existing data-driven deep reinforcement learning methods.