Autonomous driving decision-making control algorithm based on hierarchical reinforcement learning

In the decision-making model of autonomous driving based on learning, the reinforcement learning model has the problems of slow convergence and single application scenario, while the imitation learning model has the problem of poor generalization. In order to solve this problem, a two-layer reinforcement learning framework is proposed to replace the decision layer and control layer in the autonomous driving task. The decision layer divides the driving behavior into lane keeping, left lane change and right lane change. After the decision layer selects the corresponding behavior, it completes the behavior by changing the input of the control layer. Then, a new method for training the control layer, RL_COE (Reinforcement learning combined with online experts), is proposed by combining reinforcement learning and online experts. Finally, a highway simulation environment is built in Carla to verify the proposed algorithm and compare it with the reinforcement learning baseline algorithm. The results show that this method greatly improves the convergence speed and stability of the algorithm and can better complete the driving task.