Attention-Based and Rule-Guided Offline Reinforcement Learning for Highway Autonomous Driving

Lane changing decision-making on highway driving is a challenging task in autonomous driving research. To overcome this challenge, this paper proposes a novel method that integrates Batch-Constrained Q-learning (BCQ) with self-attention mechanisms for offline learning in highway lane change tasks. This method uses a rule-based pre-training strategy to generate diverse high-quality datasets, incorporating both high-quality and noisy data. The self-attention mechanism enhances the agent's ability to focus on the most relevant features of the environment, such as surrounding vehicles and lane boundaries, thereby improving decision-making. Experimental evaluations conducted in a simulated highway environment demonstrate that the proposed method significantly outperforms alternative approaches in terms of lane change success rate and driving efficiency. The results show that combining BCQ with self-attention mechanisms leads to more stable and effective driving strategies, making it a promising solution for real-world autonomous driving challenges.