Fast Integral Terminal Sliding Mode Control for UUV Trajectory Tracking Based on Deep Reinforcement Learning

ABSTRACT For the trajectory tracking control problem of Unmanned underwater vehicles (UUVs), a two‐stage controller based on deep reinforcement learning is proposed, addressing the limitations of existing sliding mode methods in convergence speed. First, considering that the existing integral terminal sliding mode surface has a slow convergence speed when it is away from the equilibrium point, this paper proposes a fast integral terminal sliding mode surface that improves the convergence speed, not only when it is away from the equilibrium point, but also when it is close to it. As a first‐stage controller, this sliding mode control (SMC) can be combined with adaptive techniques to greatly enhance the speed at which the velocity error and position error on the sliding mode surface will converge to zero. Second, different from the related fast integral terminal sliding mode methods that mainly improve convergence through the design of the sliding mode surface, a second‐stage controller utilizes the Deep Deterministic Policy Gradient (DDPG) to train the agent to adjust the first‐stage controller parameters in real time to improve overall convergence speed which is primarily reflected in a reduction of the time it takes for the state of the system to reach the sliding mode surface. In addition, it can enhance the robustness of the controller. The control laws proposed in this paper are nonsingular and do not require a priori knowledge of external disturbances as well as parameter uncertainties. Comparing the proposed control law to the existing integral terminal sliding mode method, the proposed control law significantly reduces the convergence time of the UUV dynamics error; the simulation results demonstrate the effectiveness of the proposed control method. Finally, to further validate the effectiveness and performance of the proposed control method, trajectory tracking tests were conducted in open water, confirming the validity of the proposed approach.