Collaborative Traffic Signal Control Based on Dark Causality Discovery

Multi-agent reinforcement learning methods are currently widely used in traffic lights coordination. The goal is to optimize both individual traffic light and entire area traffic lights group. Existing studies often use the centralized training and decentralized execution (CTDE) framework to solve the problem. However, traffic signal group optimization is is essentially a multiobjective optimization problem. Since the CTDE framework has only one single reward, it is difficult to design the Pareto optimal reward. Meanwhile, traffic lights need to operate continuously in actual traffic conditions. If the communication network between traffic lights is interrupted or if certain traffic lights fail, the CTDE framework will fail completely. Therefore, using a decentralized training and decentralized execution (DTDE) framework is a more practical choice. However, under the DTDE framework, each traffic light is trained independently, which cannot achieve the above-mentioned goals. To address this issue, this paper first proposes the concept of dark causal influence graph, aiming to reveal the potential causal relationships between traffic lights. Secondly, based on the hidden causal impact relationships, we redesign the reward function so that traffic lights can understand the real impact of different control behaviors on overall traffic congestion. Meanwhile, we propose the concept of experience model based on dark causal influence. The experience model can embed the traffic edges’ endogenous dark causal influence into the learning experience of each traffic light. Based on the experience model, a priority sampling method is proposed. Finally, based on the dark causality mechanism, we propose the traffic light group coordination method Causal_DQN. In the experimental section, it is verified that Causal_DQN can obtain the Pareto optimal reward. With respect to single traffic light, the proposed method can reduce the average entrance lane queue length by 21.5% compared to other methods. With respect to multiple traffic lights, the method proposed in this paper can reduce the entrance lane queue length by 23.6% compared to other methods.