Obstacle Avoidance for Unmanned Surface Vehicle by Null-Space Guidance Vector Field With Deep Reinforcement Learning

This article proposes an efficient hierarchical solution framework, null-space guidance vector field (NSGVF) with deep reinforcement learning (DRL)-based coefficient adjustment, to solve the problem of obstacle avoidance for unmanned surface vehicle (USV) with 3D-LiDAR in an unstructured environment. First, a simulation platform with high fidelity based on Virtual RobotX is developed, including the USV model, ocean environment, perception module, decision making and path planning, tracking controller, etc. With the modulated velocity commands, the NSGVF method enables the USV to navigate toward the target point and avoid obstacles, by defining the modulation matrix to quantify the influence of real-time sampling point, and introducing a null-space strategy to integrate the influence of point clouds. Because the key coefficients of NSGVF need to be set in advance with poor environmental adaptability, the DRL method with curriculum learning is then developed to find the optimum for the key coefficients. The experiment results show that the hierarchical DRL+NSGVF framework can be better applied to the task of obstacle avoidance in unstructured environments.