FairSignal: A Multiagent Reinforcement Learning Approach Considering Fairness for Multi-Intersection Traffic Signal Control

Currently, congestion has become a common urban disease. Recently, traffic control methods leveraging reinforcement learning (RL) have gained significant attention due to their capability to effectively utilize real-time traffic data. However, existing RL-based methods for multi-intersection traffic signal control (TSC) solely focus on enhancing network-level performance, which can potentially exacerbate traffic conditions at specific intersections, making it unreasonable. This balance of network-level and local intersection traffic conditions is referred to as the fairness issue in TSC. To address the aforementioned fairness issue and further alleviate congestion, a multiagent RL (MARL) algorithm, counterfactual multiagent policy gradients (COMA), which, compared to multiagent advantage actor-critic (MA2C), is characterized by its use of counterfactual advantage estimation to effectively resolve the credit assignment problem, was employed in multiintersection TSC domain. However, as an on-policy algorithm, COMA encounters limitations in training sample utilization, specifically in terms of inefficient sample usage and temporal correlation among samples. To enhance the efficiency of sample usage and to mitigate the temporal correlation among training samples, thereby improving overall training efficiency, we introduced: 1) importance sampling and 2) distributed computing. COMA, when incorporating importance sampling and distributed computing, evolved into our method, Advanced-COMA, which demonstrated effectiveness in the experiments. Additionally, a fairness-aware reward function was designed to better address fairness considerations. Finally, extensive experiments were conducted on both synthetic and real-world traffic networks, including grid and arterial scenarios. The results confirmed the superiority of our methods in enhancing traffic conditions and fairness. Some details can be found on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/caisibin/FS</uri>.