A Traffic Signal Control Method Based on Improved Deep Reinforcement Learning*

In order to solve the problem that the existing traffic signal control methods usually need to predefine the phase sequence or phase duration, a single intersection traffic signal control method based on improved deep reinforcement learning Double DQN (DDQN) was proposed. Firstly, a traffic signal control model based on deep reinforcement learning DDQN was constructed to optimize the estimated value of action-value function and the iterative process of target value. Secondly, the original one-dimensional action space was discretized into a three-dimensional action space composed of phase selection, phase duration and loss factor, and the next phase and phase green time to be executed were intelligently decided. Finally, a new loss function is designed based on the loss factor, which can dynamically adjust the loss weight of different actions according to the difference of importance. The simulation test is carried out based on the microscopic simulation platform SUMO. The experimental results show that, compared with the baseline method, the proposed method reduces the average waiting time by 28.1%, the average queue length by 29.7%, and the average number of parking vehicles by 18.8%.