Sustainable Operation of EV Battery Swapping Station with Q-Learning Considering Reneging

Battery swapping is becoming a feasible method to tackle charging issues in the expanding Electric Vehicle (EV) markets, especially in areas such as India, where two-wheelers are prevalent and three-wheelers depend on prompt service periods. In contrast to conventional methods that focus on cost efficiency and throughput, our research emphasizes Quality of Service (QoS) and sustainability through enhanced battery stock management. We represent a Battery Swapping Station (BSS) as a tandem queuing process, where congestion primarily arises during battery recharge, resulting in delays that affect swapping operations. Such delays may result in service disruptions and premature client withdrawals. To support various BSS designs, we utilize MATLAB's SimEvents toolbox to develop a customized, drag-and-drop modeling environment based on queueing theory. We make the performance measurements more realistic by integrating reneging phenomena into a Discrete Event Simulation framework. Our method implements the Q-learning algorithm with strategic reward engineering to meet a wide range of optimization goals in a variety of demand situations. This approach optimizes the usage of a limited battery stock and exhibits resilience despite limited historical data. The results indicate massive improvement in idle stock of swappable batteries (SB), enhancing stock usage and sustainability while preserving low reneging rates and attaining increased throughput with limited battery stock.