Multi-agent Deep Reinforcement Learning-based Method for Active Distribution Network Optimal Power Flow

With the large-scale integration of renewable energy generation and distributed energy resources (DERs), modern power systems face numerous new challenges, such as the intermittency of renewable energy output and the non-convex characteristic of alternating current power flow constraints. Currently, the standard method for coordinating distributed generation under network constraints is to solve the optimal power flow (OPF) problem. This paper proposes a distributed OPF method for active distribution networks (ADNs) based on multi-agent deep reinforcement learning (MADRL). First, the centralized OPF problem of ADNs, constrained by the network and containing a significant amount of renewable energy, is decoupled into a distributed OPF problem and modeled as a partially observable Markov decision process (POMDP). Then, the objective is to minimize the main grid's purchasing cost and network losses, with voltage limits imposed as constraints, leading to the derivation of the optimal control strategy. Subsequently, the proposed approach integrates a prioritized experience replay (PER) mechanism with a multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm to enhance the scalability of the algorithm by sharing effective information and efficiently utilizing experience samples. Finally, numerical examples demonstrate that the proposed algorithm outperforms the latest deep reinforcement learning algorithms and model-based solving methods, enhancing both control response speed and control strategies' quality.