Distributed secondary control for DC microgrids using two-stage multi-agent reinforcement learning

Multi-agent reinforcement learning has emerged as a promising candidate for the secondary control of DC microgrids. However, the one-stage reward function incorporating both voltage regulation and current sharing results in the significant bus voltage fluctuations and long current sharing time. To address this issue, in this paper, we propose a two-stage reinforcement learning secondary control method for DC microgrids, which can effectively suppress the bus voltage fluctuations and reduce the current sharing time. The multi-agent Proximal Policy Optimization (PPO) algorithm is utilized to regulate the current and voltage of each node in the microgrids. Specifically, a two-stage reward function based on voltage error and current error is designed, which can effectively improve the convergence speed. Moreover, an action safe mechanism is constructed to mitigate the effects of random noise and ensure the smooth operation of the DC microgrids. We have built a hardware-in-the-loop platform to verify the effectiveness of the proposed method. Experiment results show that the proposed method can effectively improve the current sharing speed and reduce the bus voltage fluctuation when compared with existing methods. • The DC microgrids secondary control was formulated as a collaborative problem. • A safe action RL mechanism was designed for DC microgrids. • A two-stage reward function was designed to accelerate the convergence.