Efficient Traffic Light and Vehicle Coordination via Heterogeneous Attention Reinforcement Learning

Coordinating traffic lights and vehicles in urban environments is essential for optimizing road traffic efficiency and reducing congestion. Traditional traffic control strategies, which typically schedule traffic lights and vehicles independently, limit the potential for enhanced overall travel efficiency. In this study, we jointly control traffic lights and vehicle management to optimize traffic flow. Specifically, a dedicated signal control process at each intersection is managed by specialized agents, while vehicles within the vicinity are modeled as agents contributing to a unified optimization process. To facilitate effective collaboration among these heterogeneous agents, we propose a novel Heterogeneous Attention Reinforcement Learning (HARL) algorithm, where Graph Attention Networks (GAT) are designed to generate weighted vectors from traffic observations across intersections, accurately capturing and utilizing real-time traffic conditions. The integration of GAT improves the representation of traffic states in complex urban scenarios, thereby enhancing traffic management efficiency. We conduct extensive experiments on various real-world urban scenarios and the simulation results show that the proposed HARL significantly reduces the queue length by more than 16.13% compared to other methods.