Trajectory tracking optimal control for unmanned surface vehicles under input saturation using actor-critic reinforcement learning

This paper proposes a trajectory tracking optimal control (TTOC) scheme for unmanned surface vehicles (USVs) with marine environmental disturbances under input saturation using an actor-critic reinforcement learning (RL) architecture. To handle the input saturation due to actuator limitations of USVs, a hyperbolic-tangent-type control input penalty function is constructed and embedded into a performance functional for the TTOC of USVs. Furthermore, an actor-critic RL architecture with adaptive neural networks is developed to learn the TTOC law for USVs, where neural network online weight update laws are devised to enable realtime optimization of the TTOC law and attenuation of marine environmental disturbances. Theoretical analyses and simulation results confirm that the proposed TTOC scheme enables USVs to track the desired trajectory under input saturation and marine environmental disturbances, while ensuring the boundedness of all signals in the closed-loop TTOC system of USVs.