Bio-Inspired Multi-Agent DQN for Maritime and Underwater Search and Rescue

With increasing complexity in maritime and underwater search and rescue missions, there is a growing need for more efficient multi-agent cooperation. Current systems face challenges in coordinating multiple Unmanned Surface Vehicles (USVs) and Autonomous Underwater Vehicles (AUVs) in dynamic environments. This study aims to apply MARL to enhance cooperation and improve search efficiency. The enhanced Deep Q-Network (DQN) algorithm leverages a bio-inspired pheromone mechanism, akin to ant behavior, to optimize the efficiency of search and rescue tasks in dynamic environments. The primary challenges addressed include handling random landmark distribution, water currents, and obstacles that affect agent performance. To adapt to these complexities, the research introduced phase differentiation in the reward system, allowing agents to transition between exploration and target discovery phases. As agents explore the environment, the pheromone mechanism guides their search by marking explored areas and assisting in cooperation between agents. With the introduction of this refined reward system and multi-agent cooperation, significant improvements in agent performance were observed. Agents displayed more efficient navigation and cooperation, especially in environments with unpredictable variables. The results indicate a notable reduction in search times and higher task completion efficiency compared to traditional models. Additionally, the study compares the DQN and MAT-DQN algorithms. While MAT-DQN enhances interpretability through the multi-head attention mechanism and has advantages in collaborative decision-making, DQN outperforms in computational speed and task completion time.