Multi-Agent Deep Reinforcement Learning with Graph Attention Network for Traffic Signal Control in Multiple-Intersection Urban Areas

Deep reinforcement learning has seen significant progress in traffic signal control. However, existing research still lacks the ability to effectively capture the correlation of road network information and the perception capability of traffic signal states. To address this gap, we propose a multi-intersection traffic signal control method that integrates a graph attention network, named the graph attention network-deep deterministic polcy gradient (GAT-DDPG) algorithm. This algorithm incorporates the restart random walk into the attention mechanism, exploring graph information through global random walks, reducing reliance on local nodes, and enhancing the model’s comprehensive understanding of graph structure features, thereby improving the modeling capability of traffic network structures. Moreover, the algorithm can automatically identify and extract key features from the complex data of the traffic network without manual intervention, adapting to different traffic network topologies, and can update and adjust the traffic signal control system in real-time to accommodate actual traffic flow and congestion situations. Experimental results indicate that the GAT-DDPG algorithm reduces average vehicle travel time significantly across three real road networks (Hangzhou and Jinan in China, and New York, U.S.) and two synthetic road network datasets. Additionally, it demonstrates optimal convergence speed and performance in these real datasets, attributed to its capability to capture global information and deeply comprehend the intricate structures of traffic networks. The research proves that this model has significant advantages in the field of traffic signal control, improving the operational efficiency of urban area intersections. Future work will incorporate additional road environment factors to better adapt to complex urban traffic.