Optimizing BSS Charging Management Considering Renewable Energy by Dynamic Multi-Step Deep RL Approach for Urban Governance

The integration of battery swapping station (BSS) with renewable energy further promotes the electric vehicle (EV) industry in urban governance. However, the complex uncertainty of the environment and limited information hinders the optimization of multi-action battery charging management in BSS. The current research also lacks consideration for renewable energy procurement actions. To address this problem, a dynamic simulation environment incorporating battery charging characteristics and transportation processes for BSS multi-action charging management is introduced. On this basis, a multi-step dynamic battery charging management model based on Proximal Policy Optimization (PPO) is proposed. The goal includes cost reduction considering both price and carbon emissions, minimizing queue lengths and avoiding wastage of charging bays. First, a prediction module estimates the real-time energy price. Then, two decision modules sequentially determine the optimal reservation number of charging bays and the procurement of renewable energy. Unlike traditional methods, the model optimizes the charging management process across multiple actions, particularly in analyzing the impact of renewable energy. Additionally, equivalent energy price coefficients are proposed as a straightforward measure to assess cost disparities between renewable energy and grid energy. The proposed model enhances decision-making efficiency compared to classical methods and demonstrates stability and performance across various scales in experiments. The simulation data of proposed model is available at https://github.com/MyGitHubCAT/GRU-MSPPO