Hierarchical Reinforcement Learning for Autonomous Parking Based on Kinematic Constraints

In recent years, the rapid increase in urban vehicles has made finding vacant parking spaces a significant challenge. Our research focuses on reverse parking maneuvers that align with driver habits and enhance safety. While end-to-end Reinforcement Learning (RL) has shown promise in autonomous driving, it struggles with parking tasks requiring flexible switching between forward and reverse actions due to large exploration space and sparse rewards. To address these challenges, this paper proposes an autonomous parking algorithm based on Hierarchical Reinforcement Learning (HRL), which decomposes complex tasks into subtasks for better exploration and accuracy. The high-level controller has two functions: building a RL-based agent that maps Bird's-eye view (BEV) images and external measurements to the ego vehicle's actions, and using the Autonomous Vehicles (AVs) chassis data to establish dynamic constraint equations for calculating the motion constraint-based parking area. The low-level controller executes specific parking maneuvers. Experimental results show that compared to baseline methods, our method achieves the highest parking success rate in both static and dynamic scenarios. Additionally, our method demonstrates smoother performance during reverse parking.