An Optimal Two-Stage Scheduling Strategy for Electric Vehicle Charging Stations Based on Deep Reinforcement Learning

Effective coordinated charging strategies are of great significance when a large number of electric vehicles (EVs) are connected to the power grid. In traditional mathematical model, the increase in the number of EVs leads to an increase in decision variables, which in turn prolongs the model solving time. Meanwhile, due to the uncertainty in the travel behavior of EV users, it is difficult to establish an accurate model. To address this issue, this paper proposes a two-stage optimization strategy based on hybrid approach. In the offline training phase, Ape-X Deep Q-Network (DQN) is employed to train the charging piles without the need for an accurate model. In the online decision-making phase, the charging station scheduling problem is formulated as a Mixed-Integer Programming (MIP) problem, with the objective of maximizing the revenue of the charging station. By approximating the action-value function as the revenue, the number of decision variables and the model solving time are effectively reduced. Experimental verification shows that, compared with traditional models and queuing theory-based scheduling methods, this strategy significantly increases the revenue of the charging station while reducing decision-making time.