Deep Reinforcement Learning for Online Reconfiguration of Active Distribution Network

The operation and control of active distribution networks (ADNs) are becoming increasingly important due to the high penetration of renewable energy (RE). The inherent uncertainty of RE can affect the stability and efficiency of ADN operations. To mitigate the inherent uncertainty and rapid variability of the high RE penetration in ADNs, this article uses an online ADN reconfiguration (ADNR) approach to ensure swift responses to RE fluctuations. Unlike traditional deep reinforcement learning (DRL)-based methods, which typically model the ADNR as a Markov decision process (MDP) and rely on historical ADN data to train the DRL agent, this approach may lead to a mismatch between the MDP's characteristics and the actual ADNR and pose challenges in handling scenarios that do not exist in the training data. To address this issue, this article proposes an online-offline DRL framework for online ADNR. Initially, during the offline stage, ADNR is formulated as a state-driven Markov decision process, which incorporates the operational characteristics of the ADN. Following this, a state-driven proximal policy optimization (SD-PPO) algorithm is proposed to enhance the generalization capability of DRL. In the subsequent step, we present the optimized action proximal policy optimization (OA-PPO) algorithm, which performs personalized training based on SD-PPO to further improve DRL performance in the online stage. The proposed approach is applied to three IEEE ADN systems. Numerical results demonstrate the effectiveness of our approach in reducing power loss and enhancing RE accommodation. Furthermore, detailed comparisons with other DRL and traditional ADNR algorithms confirm the superior computational performance of our proposed method.