Voltage Control for Distribution Networks: A Deep Reinforcement Learning Approach Improved with Option-Critic and Gated Recurrent Unit

Reinforcement Learning (RL) is widely considered an effective tool for solving real-time control problems in power systems. However, with the increasing penetration of distributed generation dominated by renewable energy sources in distribution networks, the uncertainty of generation and load, along with the diversity of controllable resources, has significantly increased the dimensionality of voltage control in distribution networks. This results in the policy degradation issue in traditional RL methods when faced with high-dimensional decision spaces in voltage control scenarios. To address this challenge, this paper proposes an Option-based Deep Reinforcement Learning (DRL) algorithm-Gated Recurrent Units Option Critic (OC-GRU). This algorithm combines the hierarchical policy structure of the option-critic framework with GRU, leveraging the strengths of GRU in time-series feature extraction, and uses the OC framework to learn the voltage control policy for the power system. Specifically, OC-GRU effectively extracts key state information from the power system using the GRU network, and optimizes both high-level options and low-level action policies through the option-critic framework, significantly reducing the action space dimensionality in the voltage control decision process. The experimental results show that the OC-GRU algorithm outperforms the benchmark DRL algorithm during training.