Map-less Navigation Algorithm for Autonomous Vehicles Based on Deep Reinforcement Learning

This paper focuses on the map-less navigation problem of autonomous vehicles based on deep reinforcement learning, and proposes a map-less navigation method for autonomous vehicles based on an improved Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. Aiming at the problems of navigation success rate, exploration performance, and training time of existing map-less navigation algorithms based on deep reinforcement learning, the following innovations are used to optimize the above performance: ① Optimize the neural network structure of the TD3 algorithm to enhance the exploration ability of autonomous vehicles in complex environments. ② Construct a composite reward function to integrate dense rewards and sparse rewards, which significantly speeds up the training speed of the algorithm. Finally, the algorithm in this paper only needs 12% of the training amount of the comparison algorithm to achieve the same success rate. A comprehensive test environment and a special test environment were built in a simulation environment for comparative experiments. The results show that the navigation success rate of the algorithm in this paper is increased by 11.80% in the comprehensive test environment; the obstacle avoidance success rate is increased by 40% and 70% in the special test environment, and the exploration success rate is increased by 100%. In the test of real complex environment, the navigation algorithm is not adjusted, and it can effectively drive the autonomous vehicle to perform map-less navigation. The navigation effect and portability of the algorithm are verified.