A Real-Time Digital Simulator Accelerated Reinforcement Learning Training Environment for Power System Frequency Dynamics

Ahstract-The growing integration of distributed energy resources (DERs), power electronic devices, and flexible loads in power systems is increasing the complexity and uncertainty in the modern power grid, bringing new challenges in its operation and control. However, with the recent advancement in artificial intelligence techniques, reinforcement learning (RL) has received increasing research interest. It has shown its potential to ensure reliable operation and improve the resilience of the power grid. Effective RL training requires significant interaction with an environment which can be difficult in the case of large-scale power system models due to long simulation duration. To address this issue, this paper proposes a co-simulation framework involving real-time digital environment simulation for accelerated training of RL agents. The proposed co-simulation framework is tested for providing fast frequency response (FFR) to the microgrid model and analyzed based on its training speed compared to another RL training approach based on an equivalent C-code environment model interface. Results show that the proposed cosimulation framework can speed up the RL training process by approximately 17 times while generating optimal RL agents.