Single degree of freedom control based on deep reinforcement learning for underwater unmanned vehicle

This paper proposes a two-stage controller based on deep reinforcement learning for an unmanned underwater vehicle under conditions of parameter uncertainty and external measurement noise. The feedback linearization technique is first applied to linearize the dynamic model, and then this is used to design the control law, the integral term is added to expand the control law into a Proportional Integral Derivative (PID) control method, which is used as the first stage controller. The second stage controller is to realize parameter regulation, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to train the agent in response to the controller’s need to adjust gain parameters in real time to cope with external interference. Moreover, in order to minimize unnecessary exploration at the beginning of the training process, the previous excellent manual experience is combined with the training process. Compared to the original DDPG algorithm, this algorithm exhibits a faster convergence rate. To verify the performance of Feedback Linearized DDPG PID Control (FLDDPGPIDC), three advanced methods were compared with the method. Results indicate that tracking performance is significantly improved in the presence of external measurement noise and dynamic parameter perturbations.