Employing the Mean-Field Approximation Technique in Multiagent Reinforcement Learning to Control Signalized Intersections in an Urban Network

Traffic signal control is a difficult task to ensure the performance of traffic networks in major cities around the world. Multiagent reinforcement learning (MARL) is a promising approach for traffic light management. As the number of agents increases, the learning process becomes impossible because of the curse of dimensionality and the interactions between agents. To solve this, we introduce a novel decentralized MARL-based approach combining the double deep Q network with the mean-field approximation technique (MFA). Our model eliminates the overestimation problem of the traditional DQN by using dual estimators during training. In addition, it also reduces model complexity by using the MFA to approximate the interaction within the population of agents as the interaction between a single agent and the average effect from neighboring agents. Our proposed method is compared against other algorithms to test its effectiveness. This study also provides an analysis of the influence of using different traffic generation tools (OD2Trips, DUArouter, Marouter, and DUAIterate) on the model performance. Experimental results demonstrate their effectiveness and robustness over other algorithms in terms of waiting time, average speed, and queue length.