Joint control of traffic signal phase sequence and timing: a deep reinforcement learning method

Recent advances in artificial intelligence have opened up new possibilities for optimizing traffic operations. In this study, a novel deep reinforcement learning-based traffic signal control strategy is proposed to jointly optimize phase sequence and signal timing, allowing for more efficient and flexible signal control. A comparison between the proposed approach and two traditional methods, Webster and MaxPressure, is conducted to highlight the advantages of AI-empowered signal control. Different techniques in state representations and action selections are explored to enhance the performance of the DRL-based agent. Results from simulation experiments indicate that the 3DQN framework with prioritized experience replay outperforms other methods by at least a 7.56% queue length reduction. Additionally, the combined state representation of macroscopic feature-based and microscopic cell-based information presents a valuable enhancement for model performance. The ablation experiments demonstrate that considering microscopic information only leads to a 2.44% increase in queue length compared to the proposed method with combined micro-level and macro-level information. However, depending only on the macroscopic feature-based representation, it fails to converge during the training session. Furthermore, the proposed joint control method reduces queue length by 6.37% compared to the phase switching control method, while the single-agent model that optimizes the phase duration performs even worse. Hopefully, this study can offer references for future research in deep reinforcement learning-based traffic signal control schemes and reveal their potential to cope with more dynamic and complex scenarios.