Data-Driven Automatic Generation Control Based on Learning to Coordinate and Teach Reinforcement Mechanism

Large-scale integration of renewable energy introduces significant random perturbations to the power system, disrupting the symmetry and balance of active power, which complicates the stabilization of the system’s frequency. Inter-regional energy cooperation plays a crucial role in maintaining the symmetry and balance of the overall power system’s active power. However, when the power system area is expanded, automatic generation control (AGC) based on reinforcement learning faces the challenge of not being able to leverage the prior experience of the original system topology to train the new area, making it difficult to quickly develop an effective control strategy. To address these challenges, this paper proposes a novel data-driven AGC method that employs a multi-agent reinforcement learning algorithm with a learning to coordinate and teach reinforcement (LECTR) mechanism. Specifically, under the LECTR mechanism, when the power system region expands, agents in the original region will instruct the agents in the newly merged region by providing demonstration actions. This accelerates the convergence of their strategy networks and improves control accuracy. Additionally, the proposed algorithm introduces a double critic network to mitigate the issue of target critic network value overestimation in reinforcement learning, thereby obtaining higher-quality empirical data and improving algorithm stability. Finally, simulations are conducted to evaluate the method’s effectiveness in scenarios with an increasing number of IEEE interconnected grid areas.