Multi-Underwater Target Interception Strategy Based on Deep Reinforcement Learning

In the context of multiple autonomous undersea vehicles(AUVs) executing underwater target interception missions, AUVs are required to make precise decisions based on both enemy and partner information, navigating the dual challenges of competition and cooperation. Most existing research typically focuses on single-target interception in simple environments and lacks a detailed exploration of collaborative mechanisms for multi-target interception mechanisms in complex environments. Therefore, this paper proposed a multi-agent deep reinforcement learning framework for AUVs to learn interception strategies in environments with complex obstacles and time-vary ocean currents, with a focus on cooperation in many-to-many game scenarios. First, a hierarchical maneuvering framework was introduced to improve the decision-making ability of AUVs through a three-layer loop structure. Next, the multi-agent proximal policy optimization algorithm was used to construct a scalable state and action space and design a compound reward function, enhancing interception efficiency and cooperation of AUVs. Finally, a population expansion–curriculum learning approach was incorporated within a centralized training and distributed execution architecture to help AUVs master generalizable cooperation strategies. Training results show rapid convergence and high success rates of the proposed interception strategies. The simulation experiments show that the trained AUVs can use the same set of models in multiple population configurations to effectively intercept multiple intruding targets through cooperation while avoiding obstacles.