Fuzzy Actor–Critic Reinforcement Learning for Unmanned Surface Vessels Flexible Tracking Formation Control

Unmanned surface vessels (USVs) play vital roles in marine operations, including environmental monitoring, resource exploration, and rescue missions. However, effective tracking formation control of USVs in complex marine environments is complicated by wind and wave interference and unexpected surface disturbances. This study proposes a novel actor-critic reinforcement learning control scheme, integrated with a fuzzy logic system (FLS), to address USV tracking and formation control problems in narrow waterways. Specifically, an FLS-based observer is designed to estimate unknown parameters, such as USV mass and speed, and environmental disturbances, and an actor–critic reinforcement learning controller is developed to allow for tracking and formation control of multiple USVs. The stability of the proposed control scheme is formally analyzed. The numerical results demonstrate the effectiveness of the controller, showcasing its ability to maintain the USV formation when tracking riverbanks rendered dynamic by environmental disturbances and obtains 24.48% reduction in trajectory length when compared with the existing controllers. This improves the autonomous tracking and formation capabilities of USVs in constrained and uncertain environments.