Autonomous Driving Decision Algorithm based on Hierarchical Dueling Double Deep Q-Network

The sparse reward in traditional reinforcement learning will lead to poor training effect of autonomous vehicles. This paper proposes a hierarchical automatic driving decision method based on reinforcement learning. The multi-modal fusion of camera images and radar data ensures the state input of autonomous vehicles. The action space is divided into high-level options and low-level actions. The high-level option network and the underlying action network are built respectively. The vehicle interacts with the environment to learn the optimal driving strategy. Through simulation experiments, compared with DDDQN and DQN algorithms, the average speed of autonomous vehicles based on HDDDQN increases by about 4.48%, the average number of lane changes decreases by about 4.54%, and the average number of overtaking increases by about 18.6 %. Therefore, the safety and driving efficiency of automatic driving are significantly improved.