A Proximal Policy Optimization-Based Controller for Enhanced Power Sharing in Microgrids

This paper introduces a Proximal Policy Optimization (PPO)-based virtual impedance (VI) controller to enhance both power sharing and system response under disturbances in inverter-interfaced microgrids. Traditional droop control methods often face challenges due to variations in feeder impedance, which degrade performance. The proposed controller continuously updates its policy based on changes in the operating environment. The control problem is modeled as a Markov Decision Process (MDP), in which the state and action spaces are explicitly defined, and a carefully designed reward function, satisfying system criteria and constraints, guides the learning process toward achieving the desired transient and steady-state performance. By leveraging PPO, the controller improves upon traditional methods by reducing the need for manual tuning and offering better adaptability to varying operating conditions. The performance of the proposed controller is evaluated in both islanded and grid-connected modes, using batteries with capacities of 1 MW, 125 kW, and 100 kW. The results demonstrate that the PPO-based VI controller improves power-sharing accuracy and provides better response to disturbances across different scenarios compared to the conventional controller. To validate the performance of the proposed method, an assessment is conducted on the system frequency using key metrics, including Root Mean Square Error (RMSE), Integral of Absolute Error (IAE), Integral of Squared Error (ISE), and Integral of Time-Weighted Squared Error (ITSE). The PPO controller consistently achieves the lowest errors across all scenarios compared to the conventional controller, with the IAE reduced by 27% in islanded mode and 36% in grid-connected mode.