Multithreaded Asynchronous Deep Reinforcement Learning With Multisensor Fusion for Robot Collision Avoidance

To develop a safe and efficient navigation system of robotic vehicles in dynamic scenes, a new collision-avoidance method using deep reinforcement learning (DRL) is presented. First, a novel method of DRL based on multithreaded asynchronous proximal policy optimization (MAPPO) is developed. It can convert expensive online calculation into an offline training process, improving the sample efficiency during policy learning. Then, a multisensor fusion measurement (MSFM) method is presented by the combination of global reference path (GRP), laser scanner measurement (LSM), and motion energy (ME), to observe the state space of environment to maximum extent. By multireward refining at each timestep, the sparsity of rewards is avoided. On this basis, a collision-avoidance neural network (CANN) fused in multiscale and multilevel is devised to generate high-quality obstacle features, which can enable the MAPPO to master collision threat effectively. Besides, a premature collision prediction (PCP) module supervised by GRP is devised as an auxiliary task to learn high-level feature representation to further improve the safety during robot collision avoidance. Finally, a two-stage training strategy from 2-D Stage to 3-D Gazebo is presented to realize sufficient robot-environment interaction. This way, the policy model can maximize its degree of exploration in complex dynamic scenarios. Extensive navigation experiments are conducted on the complex simulation and real-world scenarios with a variety of obstacles, along with multiple comparative experiments to testify the effectiveness and robustness of our approach in robot collision avoidance. Experiment results reveal that our method can make farsighted navigation decisions in complex dynamic environments to dodge collisions successfully while moving toward the goal.