Mobile Robot Collision Avoidance Based on Deep Reinforcement Learning With Motion Constraints

Deep reinforcement learning (DRL), which integrates neural networks with reinforcement learning algorithms, plays a crucial role in enhancing autonomous robot collision avoidance and navigation in various environments, including industrial warehouses, hospitals, urban pedestrian zones, and airport terminals. However, existing research has encountered limitations, such as reliance on conventional algorithms, the need for multi-sensor data fusion, and application in overly simplified or non-random scenarios. To address these challenges, this paper presents a novel end-to-end method that implicitly integrates the mobile robot's motion constraints to help the agent navigate and avoid obstacles in complex environments. By designing a novel observation space and a Long Short-Term Memory module, our DRL framework can output two continuous actions, which are linear and angular velocities for collision avoidance in dense environments, by only using the depth image from the onboard RGBD camera. To remove the assistance from conventional algorithms, we have designed a novel reward function that takes into account the motion constraints for differential drive robots, promoting higher linear velocity and lower angular velocity, thus for efficient and smooth obstacle avoidance, especially in sparse-obstacle environments. We have extensively validated the proposed framework against state-of-the-art methods using the BARN_Dataset map set. Furthermore, various real-world scenarios, including structured, unstructured, and random environments, and moving pedestrians, have further demonstrated the effectiveness and robustness of our proposed framework.