Reinforcement Learning-based AUV Trajectory Tracking and Obstacle Avoidance Control in Wave Disturbance Environments

Trajectory tracking control and obstacle avoidance control are fundamental to the Autonomous Underwater Vehicle (AUV) in executing various tasks. However, much of the existing research has treated these two tasks separately. In this paper, a trajectory tracking control framework that integrates obstacle avoidance is proposed, leveraging deep reinforcement learning algorithms. Specifically, we enhance the Proximal Policy Optimization (PPO) algorithm by improving the clipping of policy probability ratios to improve the learning performance of the AUV during the training phase. Simulation results demonstrate that the proposed method enables the AUV to avoid collisions with obstacles while successfully accomplishing the trajectory tracking task in a disturbance environment of wave. Furthermore, the method exhibits a faster training speed compared to other reinforcement learning algorithms.