Multi-Agent Reinforcement Learning-Based Cooperative Encirclement Control of Autonomous Surface Vehicles Against Multiple Targets

Autonomous surface vehicles (ASVs) have been widely applied in ocean engineering due to their small size, low cost, and high mobility. However, more relevant encirclement control methods with many-to-one are simple and do not consider the system dynamics. This article proposes a cooperative encirclement control method for ASVs against multiple targets based on multi-agent reinforcement learning. Firstly, a dynamic target allocation algorithm is designed based on location information of both vehicles and targets, enabling vehicles to select encirclement targets in real-time according to relative distances. Subsequently, the whole encirclement process is divided into multiple stages, and a multi-stage reward function is developed based on curriculum learning to guide ASVs in completing encirclement tasks progressively, from simpler to more complex scenarios. Then, the actor and critic networks incorporating long short-term memory are constructed, respectively, and a multi-agent soft actor-critic reinforcement learning algorithm is employed to train ASVs, enhancing cooperative target encirclement maneuvers. Finally, the effectiveness and superiority of the proposed method is validated through a six-on-two encirclement simulation.