Proximal Policy Optimization Reinforcement Learning-based Optimal Scheduling of Hybrid Energy Storage Systems in Microgrids

Hybrid energy storage systems (HESS) in microgrids(MGs) enhance efficiency and performance. HESS offers high energy density, rapid response, and long-duration storage, but scheduling is challenging due to PV and load uncertainty, balancing lifetime and profit, and inaccurate predictions. In this study, we propose a Proximal Policy Optimization (PPO)-based reinforcement learning approach to optimize the scheduling of HESS in MGs. Our method explicitly considers multiple factors, such as market-driven electricity price variations, prediction uncertainties, and the distinct characteristics of each storage technology. By formulating the problem as a Markov Decision Process (MDP), we design a reward function that balances economic costs, system longevity, and operational constraints. To validate our approach, we conduct experiments using an open-source MG simulation environment, testing our algorithm across four built-in scenarios. The results demonstrate that our PPO-based method is robust to prediction errors and effectively reduces the overall economic costs of the MG. Additionally, our approach optimizes the utilization of each storage component, extending the system’s lifetime while achieving significant cost savings.