Adaptive Traffic Signal Control Based on A Modified Proximal Policy Optimization Algorithm

Reinforcement learning has been shown to effectively optimize traffic signal timing. This study focuses on reducing vehicle fuel consumption and introduces an adaptive traffic signal control model, termed MPPO_TSC. This model incorporates various algorithmic enhancements over the standard Proximal Policy Optimization (PPO) framework and integrates both Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks to improve the capability of deep neural networks in extracting features from dynamic traffic flows. Additionally, to overcome the challenge of agents being unable to access real-time vehicle fuel consumption data, this study devises an approximate method for estimating differential fuel consumption and develops a novel reward function accordingly. Experimental validation was carried out on the SUMO simulation platform, with tests conducted in both simulated and real-world intersection scenarios. The results indicate that MPPO_TSC achieves reductions in average fuel consumption per vehicle and decreases in average waiting time compared to the four control strategies.