HAPS-PPO: A Multi-Agent Reinforcement Learning Architecture for Coordinated Regional Control of Traffic Signals in Heterogeneous Road Networks

The increasing complexity of urban traffic networks has highlighted the potential of Multi-Agent Reinforcement Learning (MARL) for Traffic Signal Control (TSC). However, most existing MARL methods assume homogeneous observation and action spaces among agents, ignoring the inherent heterogeneity of real-world intersections in topology and signal phasing, which limits their practical applicability. To address this gap, we propose HAPS-PPO (Heterogeneity-Aware Policy Sharing Proximal Policy Optimization), a novel MARL framework for coordinated signal control in heterogeneous road networks. HAPS-PPO integrates two key mechanisms: an Observation Padding Wrapper (OPW) that standardizes varying observation dimensions, and a Dynamic Multi-Strategy Grouping Learning (DMSGL) mechanism that trains dedicated policy heads for agent groups with distinct action spaces, enabling adequate knowledge sharing while maintaining structural correctness. Comprehensive experiments in a high-fidelity simulation environment based on a real-world road network demonstrate that HAPS-PPO significantly outperforms Fixed-time control and mainstream MARL baselines (e.g., MADQN, FMA2C), reducing average delay time by up to 44.74% and average waiting time by 59.60%. This work provides a scalable and plug-and-play solution for deploying MARL in realistic, heterogeneous traffic networks.