A Phased Training Method for Stabilizing the Training Process of Power Grid Voltage Control Agents with Deep Reinforcement Learning

In recent years, renewable energy sources such as photovoltaic power generation systems (PV) have been rapidly integrated into power grids worldwide. The increased penetration of renewable energy resources has made it more challenging to maintain proper voltage levels using the conventional Load Ratio Transformer (LRT) tap-changing method, given the rapid generation fluctuations caused by changes in weather conditions. To address this problem, reactive power control with the power conditioning system (PCS) of PV can be used as a resource for voltage regulation. Recent studies have developed multi-timescale voltage control, utilizing short-term control by PCS and longterm control by LRT, with the support of deep reinforcement learning (DRL). Most of these methods coordinate agents operating at different control cycles through reward calculation. However, training can become unstable due to improper management of changes in each agent's strategy, potentially hindering effective voltage control of the power grid. In this paper, the authors propose a phased training method to enhance the stability of the training process for each agent performing either LRT or PCS control in DRL-based power grid voltage management. The effectiveness of the proposed method is verified through numerical simulations using a power grid model with significant PV integration.