Transfer Multi-Agent Deep Meta Reinforcement Learning Method for Load Frequency Control of Performance Market-Based Multi-Area Microgrid With Prosumers

In a performance-based frequency regulation market-based multi-area islanded microgrid, there are frequent load disturbances and tie-line power fluctuations caused by prosumers, which intensified the conflict of interests among the regulation service providers (units) in different areas. In order to address this issue, a fog computing-based cooperative load frequency control (FCC-LFC) method is proposed in this paper. This method draws on the idea of fog computing, with the units treated as independent agents to output their commands. In online application, each unit only needs to collect the state in its own area for decision-making, with no more communication required. In addition, to support the method, a transferred multi-agent deep meta policy gradient (TMA-DMPG) algorithm is also proposed in this paper that introduces transfer learning and meta-reinforcement learning techniques into a multi-agent actor critic learning framework. The meta-learner first acquires meta-knowledge through high-value demonstration trajectories, while the base learner applies transfer learning techniques to achieve fast adaptation to tasks and improve the robustness of FCC-LFC strategy. As revealed by the experiments conducted on the four-area LFC model in Sansha Island in the China Southern Grid (CSG), the proposed method is effective in reducing frequency error, generation cost and regulation mileage charge.