Addressing Sim2Real Challenges in DRL-based Car-following Policies via Federated Reinforcement Learning and Augmented State Observations

The car-following policy is crucial for autonomous driving, aiming to ensure safety and efficiency in complex traffic environments. Although Deep Reinforcement Learning has advanced car-following strategies, it encounters difficulties in generalizing across varied vehicle dynamics. This paper systematically evaluate DRL-based car-following policies and identify significant performance degradation in heterogeneity environments. This study addresses these challenges through Federated Reinforcement Learning, which facilitates collaborative model training while preserving data privacy. The FRL approach demonstrates robust generalization across diverse training environments and reward settings, improving overall policy effectiveness. Additionally, for a specific action delay environment, we improve training stability and policy performance by incorporating historical action sequences into state observations. This research provides a framework to address the Sim2Real challenges in DRL-based car-following strategies, offering valuable insights for more complex autonomous driving systems.