<i>Q</i> ‐Learning‐Based Analysis of Auction Mechanisms for Balancing Power Market in a Power System With High Integration of VRE

ABSTRACT Procuring balancing power economically in accordance with the integration of variable renewable energy sources (VRE) presents a significant challenge. To meet this challenge, control reserves for balancing power have been traded through markets in many countries. Given the experience of European countries in this regard, there are two mainstream auction mechanisms in the balancing power market: uniform‐price and pay‐as‐bid auctions. For balancing power suppliers, one of the most important issues is how to determine the proper bidding strategy in the balancing power market to satisfy their profit‐maximising goals. On the other hand, it has become a trend for various players to participate in the balancing market through Virtual Power Plants (VPPs). In this paper, the fundamental characteristics of the above two auction mechanisms for tertiary control reserves are examined using a numerical simulation for VPP players based on the recommended practice for automatic generation control (AGC30) developed by IEEJ. The bidding strategy of each VPP player in the balancing power market is modelled using the Q ‐learning algorithm to maximise the player's expected profit under the uncertainty of imbalance. The clearing points in the balancing power market with the two auction mechanisms are compared based on simulation results.