Deep Reinforcement Learning for Hybrid Traffic Control: Coordinating AI and Fixed-Time Signal Controllers in Urban Networks

Recently, adaptive intersection signal control methods based on artificial intelligence (AI) have garnered substantial research interest. As these studies progress, the practical feasibility of AI-driven intersection control has been increasingly validated, with several algorithms already implemented in pilot scenarios. Consequently, urban road networks will likely feature a hybrid configuration of traditional and AI-based intersection controllers in the foreseeable future. Nevertheless, existing studies remain insufficient regarding AI control strategies operating within such mixed-control environments. Motivated by this gap, this paper investigates two critical issues: (1) the influence of an AI-controlled intersection on the signal control effectiveness of its neighboring intersections, and (2) the surrounding control environment prerequisites required for successful AI intersection deployment. To deal with these problems, this study first proposes a novel adaptive intersection signal control algorithm for individual intersections, termed downstream green-wave aware intersection controller (DGWAIC), which integrates mechanisms recognizing and rewarding upstream and downstream green-wave conditions into the proximal policy optimization framework. Subsequently, to validate the proposed methodology, a canonical experiment featuring a central AI-controlled intersection surrounded by four traditionally controlled intersections (a “1+4” experimental scenario) was designed. Comparative analyses between the proposed method and an individual reward-oriented AI control strategy were conducted under both coordinated and uncoordinated control scenarios to explore control effectiveness across varying application contexts. The experimental results underscore two key insights. First, individually reward-driven AI intersection control strategies exhibit inherently selfish and aggressive behaviors, necessitating algorithmic consideration of the propagation effects on adjacent traditional intersections to prevent localized optimization from compromising overall network efficiency. Second, the adaptability and coordination capabilities of AI-controlled intersections remain inherently limited, indicating that deploying AI controllers within environments characterized by coordinated peripheral intersection controls further leverages the advantages of AI-based traffic control.