Deep Q-Networks Signal Control Optimization Algorithm Combined with Expert Experience

A scientifically reasonable traffic signal control scheme optimizes the allocation of road rights, effectively reduces vehicle waiting time, improves intersection traffic efficiency, and achieves more efficient utilization of road resources. With the continuous development of reinforcement learning and artificial intelligence technologies, traffic signal light control algorithms have been constantly improved and optimized. However, previous studies on optimization models of signal control schemes based on reinforcement learning mostly focused on the improvement of the learning process of agents, and paid less attention to the input of signal control schemes. To improve the traffic efficiency of intersections more effectively, this study proposes a Deep Q Network (DQN) signal control optimization algorithm combined with expert experience. This algorithm first designs a timing signal timing scheme, and then uses the DQN algorithm for training in the microscopic traffic simulation environment SUMO to obtain the optimal signal control execution scheme of the intersection. Since the traffic signal timing scheme is calculated based on the road conditions, flow direction and actual traffic volume of the intersection, the input signal control scheme is more reasonable. This study conducted experiments at the intersection of Huasui Road and Huacheng Avenue and obtained the following results: Compared with the actual signal control scheme, both the DQN algorithm and the DQN optimization algorithm combined with expert experience increased the average speed of the intersection by 7. 9%; The DQN algorithm reduced the average number of waiting vehicles by 23. 1%, and the DQN optimization algorithm combined with expert experience reduced it by 69. 2%. The experimental results show that the application of both of these two optimization algorithms can effectively improve the traffic efficiency of intersections. Among them, the DQN signal control optimization algorithm combined with expert experience performs the best among all algorithms.