Adaptive Control Strategy for the PI Parameters of Modular Multilevel Converters Based on Dual-Agent Deep Reinforcement Learning

As renewable energy sources are integrated into power grids on a large scale, modular multilevel converter-high voltage direct current (MMC-HVDC) systems face two significant challenges: traditional PI (proportional integral) controllers have limited dynamic regulation capabilities due to their fixed parameters, while improved PI controllers encounter implementation difficulties stemming from the complexity of their control strategies. This article proposes a dual-agent adaptive control framework based on the twin delayed deep deterministic policy gradient (TD3) algorithm. This framework facilitates the dynamic adjustment of PI parameters for both voltage and current dual-loop control and capacitor voltage balancing, utilizing a collaboratively optimized agent architecture without reliance on complex control logic or precise mathematical models. Simulation results demonstrate that, compared with fixed-parameter PI controllers, the proposed method significantly reduces DC voltage regulation time while achieving precise dynamic balance control of capacitor voltage and effective suppression of circulating current, thereby notably enhancing system stability and dynamic response characteristics. This approach offers new solutions for dynamic optimization control in MMC-HVDC systems.