Optimization of shunting operation plan in large freight train depot based on DQN algorithm

Shunting operation plan is the main daily work of the freight train depot, the optimization of shunting operation plan is of great significance to improve the efficiency of railway operation and production and transportation. In this paper, the deep reinforcement learning (DRL) environment and model of shunting operation problem are constructed by three elements: action, state and reward, taking shunting locomotive as the agent, the lane number of the fall-down train group as the action, the fall-down conditions of the train group as the state, and design the reward function based on the total number of shunting hooks generated after the group's descent and reorganization. The model is solved using the Deep Q network (DQN) algorithm with the objective of minimizing the number of shunting hooks, the optimal shunting operation plan can be solved after sufficient training. DQN is verified to be effective through example simulations: Compared to the overall planning and coordinating (OPC) method, DQN produces a shunting operation plan that occupies fewer lanes and produces 10% fewer total shunting hooks. Compared to the binary search tree (BST) algorithm, DQN produces 5% fewer total shunting hooks. Compared with the branch and bound (B&B) algorithm, DQN takes less time to solve, and the number of freight train removed by the coupling and slipping operations is reduced by 5.3% and 2.9%, respectively, and the quality of the shunting operation plan is better. Therefore, this paper provides a new solution for the intelligentization of shunting operations in large freight train depot.