Federated Reinforcement Learning for Traffic Signal Control: Aggregation Strategies under Intersection-Level Heterogeneity

Urban traffic signal control must adapt to demand that varies across intersections, time of day, and network topology, and centralized reinforcement learning becomes impractical when every intersection must transmit raw observations to a single learner. Federated reinforcement learning (FRL) relaxes this constraint by training local controllers and exchanging only model updates, but the evidence supporting it is dispersed across incompatible experimental settings. This article reviews ten recent studies of federated and distributed reinforcement learning for traffic signal control, together with a public benchmark for the underlying control task, and organizes them through a common comparison framework covering traffic-control setting, learning paradigm, reinforcement learning algorithm, federation topology, aggregation strategy, data heterogeneity, efficiency, privacy and security, evaluation design, and deployment readiness. The review separates federated methods from distributed and multi-agent reinforcement learning, which coordinate through peer consensus or local interaction rather than a federation server, and it traces a shift from uniform global aggregation toward partial-layer sharing, clustered and hierarchical federation, and personalized combinations of global and local models. The synthesized results indicate that model-level collaboration can accelerate convergence and reduce delay and queue length in several calibrated networks, yet the gains are conditional: fixed-time control remains competitive under light or balanced demand, shared parameters weaken on networks with irregular intersection geometry, and improvements are seldom accompanied by repeated trials, uncertainty estimates, or adversarial privacy evaluation. Progress therefore depends on task-specific aggregation design and on evaluation protocols that hold the traffic scenario constant while the federation configuration is varied.