Decoupled Volt/Var Control With Safe Reinforcement Learning Based on Approximate Bayesian Inference

Deep reinforcement learning (DRL) has proven promising for addressing the problems brought by the uncertainties of renewable energy sources in the volt/var control (VVC) problem of active distribution networks (ADNs). However, deploying a conventional DRL-based VVC model cannot simultaneously achieve optimal reactive power generation loss (RPGL) and satisfy voltage safety constraints (VSCs). To tackle the challenges, we propose an approximate Bayesian inference (ABI)-based twin-critic safe reinforcement learning (TCSRL) algorithm with a reactive power fine-tuning (RPFT) strategy. The cost optimization of violating VSCs and RPGL optimization is first decoupled to obtain a better-performing VVC policy by formulating VVC as an ABI problem. A novel TCSRL algorithm is then developed to address the ABI problem. In the algorithm, the trajectory distribution satisfying the cost threshold constraint is analytically computed with an optimality guarantee, and the VVC policy is learned toward lower RPGL. In addition, twin-critic networks are constructed to improve the algorithm robustness in estimating RPGL and the cost of violating VSCs. An RPFT strategy is finally modeled and integrated into the TCSRL algorithm to ensure a 100% voltage safety rate in the training and verification phases. The performance of the proposed algorithms is validated on different IEEE bus systems.