A Novel Formation Control Strategy for USVs With Improved DDPG: Simulation and Field Test

An efficient formation-keeping strategy is essential for unmanned surface vehicles (USVs) to achieve complex cooperation missions in the Marine Internet of Things (MIoT) system. However, traditional methods make generating an efficient strategy to adapt to different formation patterns difficult in dynamic MIoT. To address this, we enhance the deep deterministic policy gradient (DDPG) algorithm and propose a novel formation control strategy generation approach. First, we design a generic reward mechanism based on the virtual leader–follower strategy to adapt to different formation patterns, simplify the design process, and optimize the formation control. Then, we adopt the intrinsic curiosity module (ICM) to alleviate the problem of sparse rewards and the prioritized experience replay (PER) mechanism to improve the utilization of experience and accelerate the learning rate. In addition, a Gaussian noise model is integrated into the DDPG approach to simulate various external disturbances, which can improve the robustness of the generated strategy. Finally, we built a virtual simulation environment based on Unity3D and conducted field tests to verify the feasibility and superiority of our approach.