Multi-Agent Reinforcement Learning for UUV Swarm-Based Capture with Opponent Modeling

This study addresses the challenging task of capturing dynamically moving target Unmanned Underwater Vehicles (UUVs) by proposing a Multi-Agent Reinforcement Learning (MARL) method integrated with opponent modeling. The proposed method employs an opponent modeling network to predict target UUV strategies and behaviors, guiding real-time decisions of the UUV swarm. Initially, a comprehensive UUV system model is developed through kinematic analysis, classifying target strategy models based on detection capabilities. Subsequently, an opponent modeling network is designed by integrating Artificial Potential Field (APF) and Dynamic Window Approach (DWA) algorithms to optimize target escape strategies realistically. Based on predicted behaviors, MARL dynamically adjusts capture strategies for the UUV swarm. Validation through a Unity3D-based virtual simulation environment demonstrates that, compared to traditional methods, the proposed approach significantly enhances strategy prediction accuracy and improves the adaptability and capture efficiency of the UUV swarm, providing valuable insights into autonomous decision-making capabilities for UUVs.