MetaSignal: Meta Reinforcement Learning for Traffic Signal Control via Fourier Basis Approximation

Traffic signal control plans significantly impact transportation system efficiency by regulating traffic conditions at intersections. Adaptive traffic plans that can adjust to real-time road conditions are more effective as a result. Reinforcement learning succeeds at adapting strategies based on feedback derived from the environment, and is thus proficient in dealing with complex traffic scenarios that change dynamically. However, current RL methods rely on significant computational periods to obtain precise functioning mechanisms within the scenarios, posing barriers to their adoption for new scenarios. In addition to directly optimizing the RL model itself to enable fast learning from scratch, another idea is to make the model transferable or reusable with the learned experience. Given the diversity of migration scenarios, the underlying control algorithm should guarantee convergence and endeavor to be parameter-insensitive. From the above concern, we proposed MetaSignal, an efficient meta-reinforcement learning method for traffic signal control. Specifically, our approach utilizes the Fourier basis as the value function approximation in reinforcement learning, distinguishing it from methods like neural network approximation. This linear approximation offers advantages such as convergence facilitation, error bound achievement, and reduced parameter dependence. The meta-learning framework adopts a model-agnostic approach, enabling effective adaptation of the base model to the target scenario with limited training cost. Empirically, the proposed method shows promising and stable performance for traffic signal control through comprehensive comparison experiments in both synthetic and real-world traffic networks.