Cooperative Multi-AUV Data Collection Method Based on Staged Deep Reinforcement Learning With Distributed Negotiation

The complex submarine geography, variable ocean current dynamics, and challenging underwater communication conditions present significant technical challenges to the implementation of data collection systems for autonomous underwater vehicles (AUVs), which are crucial for the deployment of Internet of Underwater Things (IoUT) applications. Multi agent reinforcement learning (MARL), in which agents interact with their environment to obtain rewards by joint trial-and-error, has been thoroughly researched for the cooperation of intelligent vehicles. Nevertheless, the conventional MARL approaches for cooperative AUVs are challenging to apply due to the difficulties in information exchange and the time-varying submarine environment. Accordingly, this study proposes a novel deep reinforcement learning (DRL) approach named staged learning with distributed negotiation (SLDN). Compare to prevalent MARL approaches like centralized training with distribute execution (CTDE), the proposed method relies on staged and independent DRL and accomplishes multi-AUV cooperation tasks through staged negotiation and training, aiming to facilitate collaborative operations in dynamic underwater environments while significantly reducing communication costs. This method addresses the IoUT scenario of a cluster-structured hybrid underwater network that integrates acoustic and MI communications, and its superiority in terms of data collection capabilities is confirmed by simulations and comparative analysis.