Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System

This paper discusses the development and performance evaluation of a single-inverter-based microgrid control system using a Deep Reinforcement Learning (DRL) agent trained with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The controller is specifically designed to address unplanned transitions during fluctuating load conditions and transitions between grid-tied and islanded modes. An analysis of Final Exploitation Rates (FER) ranging from 5% to 1% was conducted to assess transient responses and steady-state performance. A comparison with a traditional Proportional-Integral-Derivative (PID) controller reveals that while the DRL agent with 2%-FER demonstrates strong performance in managing transient events, the PID controller provides superior stability in steady-state operation. Key performance indicators, including Rise Time, Settling Time, Settling Min/Max, Peak Time, and Mean Absolute Deviation (MAD), were examined. The results show that the DRL agent with 2%-FER effectively limits frequency deviations to ±1% and voltage deviations to within ±2.5% during transitions, whereas agents with FER 3−5% displayed instability with deviations of up to 2% for frequency and 4% for voltage. The PID controller, however, maintained more consistent steady-state performance, with frequency tightly distributed around 49.95−50.05 Hz and voltage near 1 pu. These findings suggest that DRL-based control is better suited for handling unplanned transitions, where proactive adjustment is crucial, while PID control is more effective for steady-state conditions where a passive control approach accommodates system dynamics more naturally.