Multi-Time Scale Voltage Optimization Strategy for Distribution Networks Based on Deep Reinforcement Learning

To address the challenges posed by the traditional physics-based voltage optimization methods, which struggle with multi-resource coordinated control in distribution networks with high penetration of photovoltaic (PV) systems, this paper proposes a multi-time scale voltage optimization method for distribution networks based on deep reinforcement learning. This method considers the operational characteristics of different control devices by maintaining two agents operating on different time scales, making control decisions for discrete and continuous devices, respectively. On a long-time scale, the optimization objective is voltage limit violations, where the discrete agent controls capacitor switching by training with the Deep Q Network (DQN) algorithm. On a short time scale, the objective is voltage deviation at each node, where the continuous agent is trained using the Deep Deterministic Policy Gradient (DDPG) algorithm to determine the reactive power output of PV systems. Finally, the effectiveness and superiority of the proposed strategy in voltage optimization are verified through simulations on the IEEE 33-node system.