DRL-Based Distributed Coordination of ISO and DSOs in Bi-Level Electricity Markets

The increasing penetration of distributed energy resources has prompted distribution system operators (DSOs) at the retail electricity market level to coordinate with the independent system operator (ISO) at the wholesale market level, for greater benefits. However, interaction mechanisms between the ISO and DSOs, and impacts of prices and power injections, have not been adequately investigated in literature. This article proposes a distributed coordination framework for the ISO and DSOs across wholesale-retail (bi-level) electricity markets, considering their interactions more fairly. Moreover, to mitigate the challenges arising from the interdependence between the ISO and heterogeneous DSOs, a coupled training mechanism based on the response model is devised. This mechanism iteratively trains the ISO and DSOs by solely exchanging prices and power injections, ensuring the demand–supply balance at both retail and wholesale levels. In addition, a deep reinforcement learning algorithm is introduced for the three-stage iterative training process of heterogeneous agents. Results demonstrate the effectiveness of the proposed method and its advantages in terms of lowering energy prices, clearing of cheaper clean resources and thus, improving overall market efficiency.