Multi-AUV Path Planning Based on Hierarchical Opponent Multiactor-Attention-Critic Reinforcement Learning Framework

The increasing exploitation of marine resources and intensifying human activities have led to significant marine pollution, highlighting the critical challenge of pollutant dispersal in aquatic environments. To address this challenge, we propose a path-planning framework based on multi-agent reinforcement learning to enable rapid deployment of detection equipment to polluted sites. Specifically, we propose a hierarchical opponent multi-actor-attention-critic framework (HOMAAC) to optimize equipment delivery path planning. In HOMAAC, the hierarchical graphical attention mechanism is employed to enhance cooperative perception and interaction among entities. Opponent modeling is innovatively employed to enhance collaboration efficiency and reduce conflicts among autonomous underwater vehicles (AUVs). This modeling approach enables each AUV to dynamically adjust its motion trajectory, thereby improving the efficiency and safety of equipment delivery and path planning. The results of experiments conducted in a multi-agent underwater logistics delivery environment validate the superior performance of the proposed framework in terms of task completion rate and time spent in addressing the path planning issue, providing valuable insights for the field of intelligent underwater delivery and marine environment monitoring.