A Multiagent Reinforcement Learning-Based MAC Protocol for Clustered Multihop Underwater Acoustic Sensor Networks

Underwater acoustic channels present unique challenges, including long propagation delays and narrow bandwidth, which complicate channel access in clustered underwater acoustic sensor networks. This paper proposes a multi-agent reinforcement learning-based MAC protocol for clustered multi-hop underwater acoustic sensor networks (MARL-CMH-MAC) to address channel access issues within intra-cluster and inter-cluster transmission. The protocol employs efficient time-slot scheduling and a handshake-based channel reservation mechanism for each transmission type. A special MAC frame with adaptable slots is tailored for the clustered architecture based on the composition of intra-cluster nodes and the relaying conditions of cluster head (CH) nodes, ensuring adequate transmission time for both CH and non-cluster head (nCH) nodes. Additionally, the multi-agent reinforcement learning (MARL) model is introduced to adjust the intra-cluster access slots of nCH nodes and optimize the position of intra-cluster slots in the MAC frame, avoiding possible transmission conflicts and enhancing overall network transmission efficiency. Simulation results demonstrate that the proposed protocol is well-suited for clustered multi-hop UASNs, effectively improving the network throughput and channel utilization.