EnsembleFollower: A Hybrid Car-Following Framework Based on Hierarchical Planning and Reinforcement Learning

Car-following models have made significant contributions to our understanding of longitudinal driving behavior. However, they often exhibit limited accuracy and flexibility, as they cannot fully capture the complexity inherent in car-following processes, or may struggle in unseen scenarios due to their reliance on confined driving skills present in training data. It is worth noting that each car-following model possesses its own strengths and weaknesses depending on specific driving scenarios. Therefore, we propose EnsembleFollower, a closed-loop hierarchical planning framework for achieving human-like autonomous car-following. The EnsembleFollower framework involves a high-level Reinforcement Learning-based agent responsible for judiciously managing multiple low-level models (such as Intelligent Driver Model and Gipps model) according to the current state, either by selecting an appropriate car-following model to perform an action or by allocating different weights across all primitive models. Moreover, we propose a jerk-constrained kinematic model for more convincing microscopic traffic simulations. We evaluate the proposed method based on real-world driving data from the HighD dataset. The experimental results illustrate that EnsembleFollower yields an improved accuracy of human-like behavior and achieves effectiveness in combining hybrid models, demonstrating that our proposed framework can handle diverse driving conditions by leveraging the strengths of various car-following models.