Order-Based Heterogeneous Agents Reinforcement Learning Method With the Coordination of Distribution Network and Transmission Network for Load Frequency Control

With the continuous penetration of large-scale distributed energy resources (DERs) into the distribution network (DN), the DN has acquired the capability to participate in load frequency control (LFC). This paper proposed an orderbased heterogeneous-agent soft actor-critic method (OHASAC) to address the coordination challenges among heterogeneous controllable DERs. A neural network is applied to estimate the optimal update sequence of the heterogeneous agents. The optimal LFC problem is formulated as a partially observable Markov game (MG) considering the coordination of DN with sufficient controllable DERs and transmission network (TN). The modeling process considers the coordinated frequency regulation between battery energy storage systems (BESS) and photo-voltaics (PV) under variable irradiance conditions. Simulations in the coordinated environment of the distribution network (DN) and transmission network (TN) demonstrate that the proposed method can effectively manage various distributed generation resources while achieving superior generalization and scalability.