Traffic flow signal control and optimization based on multi-agent deep reinforcement learning

Facing the escalating challenge of the widening gap between traffic supply and demand, optimizing urban traffic signal timing controls has emerged as an effective solution to enhance road capacity. This paper proposes a Multi-Agent Double Deep Q-Network (MADDQN) to address the common supervision problem of data transmission between adjacent signals (agents) when managing regional traffic signals at different intersections. This model enhances decision-making by allowing for the sensing of environmental and phase states at both upstream and downstream intersections. Our proposed MADDQN utilizes a deep double-Q network, relying on the evaluation values in the network to determine weights and calculate target values. Furthermore, it records network parameters at each step during the iterative training process and uses these parameters from a selected neighboring step to fit the state evaluation value. This process enables the construction of an updated target value based on the average of these weightings, thereby improving the stability of the network model's training. The experimental results show that the proposed method effectively reduces the vehicle congestion, improves the accuracy of signal phase switching strategies, effectively coordinates the selecting actions at various intersections, and improves the traffic efficiency of traffic flow within the area. Even in few-shot scenarios samples/intersection), the algorithm can maintain its generalization and robustness.