Deep reinforcement learning Lane-Change Decision-Making for autonomous vehicles based on motion primitives library in hierarchical action space

Deep Reinforcement Learning (DRL) is capable of learning a policy with great scene adaptation ability through interactions with the environment, and has application potential in the field of autonomous driving. However, using DRL to directly control the vehicle motion command is easy to lead to fluctuation and non-smoothness. Discrete DRL decision-making method can generate stable behavior but lose some maneuverability. Thus, the trade-off between flexibility and stability to enhance the performance of DRL policy is an important issue. In this paper, a Deep Reinforcement Learning Decision-Making algorithm based on Motion Primitives Library (MPL) in hierarchical action space is proposed to provide flexible and reliable maneuvers for autonomous driving. The upper action space contains discrete lane-changing targets, and the lower action space is mapped as a motion primitives library. In addition, model predictive control method (MPC) based on the vehicle kinematics model is used to optimize the motion primitives instantly. The performance of the proposed method is evaluated through highway simulation. The results show that the method can make the autonomous driving lane changing process safer, more efficient and more comfortable.