Research on Cooperative Pursuit Strategy of Multiple UUVs Based on Deep Reinforcement Learning in Complex Dynamic Environments

Cooperative pursuit of multiple unmanned underwater vehicles (UUVs) is a typical adversarial task in the fields of underwater security and target interception. However, inherent underwater challenges such as communication delay, ocean-current disturbances, limited sonar detection range, randomly distributed obstacles, and maneuvering evasive targets may lead to state information lag, difficulty in cooperative decision-making, unstable pursuit formation, and complex dynamic obstacle avoidance. To tackle the above issues, this paper establishes a hierarchical UUV model that combines horizontal-plane kinematics, simplified surge-yaw dynamics, ocean-current disturbance terms, and low-level actuator saturation constraints within a deep reinforcement learning framework. Secondly, a PER-TD3-based escape strategy learning method is proposed to construct a maneuverable evasive target model; the trained evader maps local sonar, pursuer-relative, obstacle, and current observations into bounded speed and yaw-rate commands, thereby providing an adversarial target for subsequent pursuit training. Thirdly, aiming at the decision-making lag caused by communication delay and local observation, an LSTM-PMADDPG single-target pursuit method is presented as a temporal baseline. Finally, in view of the interference of dynamic obstacles and the difficulty of multi-target assignment, an LSTM-MATD3 multi-target pursuit algorithm is constructed to realize autonomous multi-target allocation, dynamic obstacle avoidance, and stable encirclement.