Perceptual Uncertainty-Aware Motion Planning for Autonomous Driving Based on Adaptive Heuristic Reinforcement Learning

Autonomous driving has become an inevitable trend in automotive development. Reinforcement Learning (RL) is extensively used in autonomous vehicle motion planning, demonstrating good generalization but facing challenges of long training times and lack of consideration of uncertainty. To address these challenges, an Adaptive Heuristic Reinforcement Learning (AHRL) approach is proposed. First, this study improves upon the Dueling Double Deep Q Network (D3QN) algorithm by proposing the Adaptive Heuristic Dueling Double Deep Q Network (Adapt-HD3QN) algorithm. Specifically, heuristic functions from search-based planning algorithms are incorporated into the RL reward terms to guide heuristic learning and enhance learning efficiency. Additionally, considering the uncertainties in real-world driving environments, such as the movement of other traffic participants and building occlusions, a Mixed Artificial Potential Field (Mix-APF) is implemented to address static and dynamic obstacle avoidance. Furthermore, potential collisions between the autonomous vehicle and other vehicles in occluded areas are modeled as a zero-sum game, with a Dynamic Bayesian Network (DBN) used for prior modeling of potential vehicles, aiding in constructing the potential vehicle’s forward hidden set. Finally, a signal-free intersection scenario, a typical crash-prone road type, is constructed on CARLA, incorporating static obstacles and other traffic participants. Experimental results demonstrate that the proposed Adapt-HD3QN algorithm exhibits superior safety, training efficiency, and traffic efficiency in scenarios with road environment perception uncertainties caused by occlusion.