Considering Energy Storage Optimization Control Based on Deep Reinforcement Learning under Two-Part Tariff

In recent years, the increasing proportion of renewable energy and energy storage in microgrids has posed challenges for their optimal control. To reduce the operational cost of microgrids under the two-part tariff system, an optimization control algorithm for microgrids based on deep reinforcement learning is proposed. This method models the microgrid optimization control problem as a Markov Decision Process (MDP), designs a reward function considering both energy charges and demand charges, and employs the Proximal Policy Optimization (PPO) algorithm to train on samples generated by a photovoltaic and load simulation system, achieving optimal control of microgrids under the two-part tariff. Finally, simulation verification is conducted for typical seasonal operating conditions. The experimental results show that the proposed deep reinforcement learning optimization control strategy reduces the annual microgrid operational cost by $1.5 \%$ compared to the timed charge-discharge strategy, reduces the monthly maximum demand by nearly $30 \%$ except in summer, and reduces the monthly maximum demand by nearly $8 \%$ in summer. The results fully demonstrate the effectiveness of the proposed method, providing a new path and possibility for the economic optimization control of “source-grid-load-storage” integrated microgrids.