Multi-microgrids Energy Management Based on Personalized Federated Reinforcement Learning

This study aims to solve the energy management problem of multi-microgrids (MMGs) through a personalized federated reinforcement learning (PFRL) approach to minimizing the operational cost of microgrid while ensuring a good user experience. In this study, we introduce the Partially Observable Markov Decision Process (POMDP) framework to describe the energy management problem of MMGs systems, instead of using sensitive information directly, non-sensitive historical data including photovoltaic power output, load power, outdoor temperature, etc. In order to solve the data silos problem among the microgrids and to promote knowledge sharing, PFRL combines locally trained and personalized models. PFRL allows each microgrid to develop personalized reinforcement learning models based on its own environmental characteristics, and global parameter aggregation through a federated learning (FL) mechanism to overcome the problem of locally optimal solutions. Simulation results obtained under heterogeneous microgrid environments indicate the advantage of PFRL and the comparisons with other algorithms are carried out to verify the effectiveness of PFRL.