EA-D3QN: An Environment-Adaptive Train Trajectory Optimization Approach for Urban Rail Transit

Urban rail trains frequently encounter various unexpected situations during operation, rendering the original speed trajectory inapplicable to the current scenario. In response to this challenge, this paper proposes a real-time optimization approach which adaptively adjusts train trajectories according to environmental changes. The approach, named Environment-Adaptive Double-Dueling-Deep Q-Network (EA-D3QN), consists of three parts: intelligent train operation (ITO) model, environment interaction, and value learning. First, the ITO model is established based on the Markov decision process. Its state encompasses external environmental information and train operation information to address various unexpected situations. Then, the environment interaction process is designed, proposing an adjustment policy for optimizing control commands to generate high-quality experience data that adapts to environmental changes. Finally, by constructing double-dueling-deep Q-networks to learn the value function, real-time generation of optimized speed trajectories is achieved. The effectiveness and environmental adaptability of the proposed approach are verified via simulations with the actual data from a Beijing Subway operating line. An optimal combination of hyperparameters is determined via sensitivity analysis.