Management of Multiple Microgrids Considering Soft Loads Based on Federated Reinforcement Learning

As the energy transition advances, multi-microgrid systems are being widely applied due to their advantages in effectively accommodating clean energy and achieving zero-carbon power supply within certain areas. However, due to the instability and intermittency of renewable energy units within these systems, intelligent regulation of the output of various units in multi-microgrid systems has become an urgent issue to address. Reinforcement learning (RL), benefiting from its strength in making decisions through interaction with the environment, has emerged as one of the effective solutions to this problem. Yet, the generalization performance of the model places demands on training data and diversity. To this end, a reinforcement learning model based on a federated learning framework is proposed, taking into account soft loads on the user side and power losses within the network. After modeling the multi-microgrid system, the specific training process can be briefly summarized as follows: first, mixed strategies are employed at each local end to train the RL models within individual microgrids; then, parameters required for model updates are transmitted to a global server for federated learning; subsequently, the models of each microgrid are updated based on the global server model. Finally, simulations and analyses are conducted on a small-scale multi-microgrid example composed of three microgrids, verifying the effectiveness and superiority of the algorithm, as well as its enhancement of market economic benefits.