Deep Reinforcement Learning-Based Voltage Regulation Using Electric Springs in Active Distribution Networks

The increasing penetration of distributed generation in active distribution networks (ADNs) introduces significant voltage regulation challenges due to the intermittent nature of renewable energy sources. Electric springs (ESs) have emerged as a cost-effective alternative to conventional FACTS devices for voltage regulation, requiring minimal energy storage while providing fast, flexible reactive power compensation. This paper proposes a deep reinforcement learning (DRL)-based approach for voltage regulation in balanced active distribution networks with distributed generation. Electric springs are deployed at selected buses in series with noncritical loads to provide flexible voltage support. The main contributions of this work are: (1) a novel region-based penalized reward function that effectively guides the DRL agent to minimize voltage deviations; (2) a coordinated control strategy for multiple ESs using the Deep Deterministic Policy Gradient (DDPG) algorithm, representing the first application of DRL to ES-based voltage regulation; (3) a systematic hyperparameter tuning methodology that significantly improves controller performance; and (4) comprehensive validation demonstrating an approximately 40% reduction in mean voltage deviation relative to the no-control baseline. Three well-known continuous-control DRL algorithms, Twin Delayed Deep Deterministic Policy Gradient (TD3), Proximal Policy Optimization (PPO), and DDPG, are first evaluated using the default hyperparameter configurations provided by MATLAB R2022b.Based on this baseline comparison, a dedicated hyperparameter-tuning procedure is then applied to DDPG to improve the robustness and performance of the resulting controller. The proposed approach is evaluated through simulation studies on the IEEE 33-bus and IEEE 69-bus test systems with time-varying load profiles and fluctuating renewable generation scenarios.