Shaped MDPs for Safe Reinforcement Learning of Reactive-Current Injection During Voltage Sags in Inverter-Based Distributed Generation
Renewable energy sources (RESs) challenge grid stability due to their intermittency, particularly during transient disturbances. Voltage sags can rapidly degrade system performance, requiring prompt reactive-current injection to support voltage recovery. Fault ride-through strategies are therefore essential to maintain stable operation of inverter-based systems while complying with grid-code requirements. Traditional analytical and optimization-based methods rely on accurate models and adapt poorly to dynamic conditions. This study proposes a data-driven reinforcement learning (RL) controller, specifically Soft Actor–Critic, for voltage sags in three-phase inverter-based systems, without solving optimization problems. To satisfy grid-code requirements, the proposed method incorporates state-dependent constraints directly into the Markov decision process through transition-function shaping. This approach preserves the original RL algorithm and avoids tuning penalty coefficients. The RL controller follows the same trend as the optimization benchmark across different reference-power levels. The results show that under various severe voltage-sag conditions, the mean absolute error of the RL predictions relative to the optimization results remains below 0.5A. The controller also generalizes across a broader range of operating conditions and uses a simpler training structure than previous RL-based studies. In addition, laboratory experiments further demonstrate its effectiveness in supporting the grid during voltage sags.
Authors
- Josué Duarte (ORCID: https://orcid.org/0000-0002-9532-4491)
- Antonio Camacho (ORCID: https://orcid.org/0000-0002-0673-6452)
- Cecilio Ángulo (ORCID: https://orcid.org/0000-0001-9589-8199)
- Mohana Fathollahi (ORCID: https://orcid.org/0009-0009-5015-743X)
Institutions
- Universitat Politècnica de Catalunya (ES)
Publication Details
- Journal
- Electronics
- Published
- 2026-11-12
- DOI
- https://doi.org/10.3390/electronics15225247
- Primary Topic
- Microgrid Control and Optimization
- Type
- article
- Field-Weighted Citation Impact
- 0.00