Real-Time Dispatching and Operation Management of Battery Swapping Stations With Uncertain Demand via Deep Reinforcement Learning

Electric vehicle charging stations are widespread but suffer from long charging times. In contrast, battery swapping stations have gained attention due to their efficiency and small footprint. However, there is a lack of extensive discussion on their layout, operation modes, and scheduling algorithms. This paper discusses the layout-dispatching-scheduling model of battery swapping stations and super battery swapping stations under centralized charging and unified dispatch. Considering battery swapping stations service time and electric vehicles queuing, a queuing-aware location-routing problem is proposed and solved using Gurobi. This study tackles the uncertainty in electric vehicle spatio-temporal dispatch by formulating the battery scheduling process between super battery swapping stations and battery swapping stations as a vehicle routing problem with time windows and uncertain demand. To address this challenge, the study proposes an adaptive routing optimization method based on an improved proximal policy optimization algorithm. Additionally, it investigates a flexible charging strategy for super battery swapping stations, where the battery charging and discharging process is modeled as a Markov decision process. To optimize operational revenue, meet demand, enable grid interactions, and contribute to peak shaving, the study employs a deep reinforcement learning approach that utilizes the twin delayed deep deterministic policy gradient algorithm. The system design is proven to be feasible and capable of meeting operational requirements.