Multi-Objective Reinforcement Learning for Virtual Impedance Scheduling in Grid-Forming Power Converters Under Nonlinear and Transient Loads

Grid-forming power converters play a foundational role in modern microgrids and inverter-dominated distribution systems by establishing voltage and frequency references during islanded or low-inertia operation. However, when subjected to nonlinear or impulsive impact-type loads, these converters often suffer from severe harmonic distortion and transient current overshoot, leading to waveform degradation and protection-triggered failures. While virtual impedance control has been widely adopted to mitigate these issues, conventional implementations rely on fixed or rule-based tuning heuristics that lack adaptivity and robustness under dynamic, uncertain conditions. This paper proposes a novel reinforcement learning-based framework for real-time virtual impedance scheduling in grid-forming converters, enabling simultaneous optimization of harmonic suppression and impact load resilience. The core of the methodology is a Soft Actor-Critic (SAC) agent that continuously adjusts the converter’s virtual impedance tensor—comprising dynamically tunable resistive, inductive, and capacitive elements—based on real-time observations of voltage harmonics, current derivatives, and historical impedance states. A physics-informed simulation environment is constructed, including nonlinear load models with dominant low-order harmonics and stochastic impact events emulating asynchronous motor startups. The system dynamics are modeled through a high-order nonlinear framework with embedded constraints on impedance smoothness, stability margins, and THD compliance. Extensive training and evaluation demonstrate that the learned impedance policy effectively reduces output voltage total harmonic distortion from over 8% to below 3.5%, while simultaneously limiting current overshoot during impact events by more than 60% compared to baseline methods. The learned controller adapts continuously without requiring explicit load classification or mode switching, and achieves strong generalization across unseen operating conditions. Pareto analysis further reveals the multi-objective trade-offs learned by the agent between waveform quality and transient mitigation.