Delay-Fluctuation-Resistant Underwater Acoustic Network Access Method Based on Deep Reinforcement Learning

The slow propagation speed of acoustic waves in water leads to significant variations and random fluctuations in communication delays among underwater acoustic sensor network (UASN) nodes. Conventional deep reinforcement learning (DRL)-based underwater acoustic network access methods can adaptively adjust their parameters and improve network communication efficiency by effectively utilizing inter-node delay differences for concurrent communication. However, they still suffer from shortcomings such as not accounting for random delay fluctuations in underwater acoustic links and low learning efficiency. This paper proposes a DRL-based delay-fluctuation-resistant underwater acoustic network access method. First, delay fluctuations are integrated into the state model of deep reinforcement learning, enabling the model to adapt to delay fluctuations during learning. Then, a double deep Q-network (DDQN) is introduced, and its structure is optimized to enhance learning and decision-making in complex environments. Simulations demonstrate that the proposed method achieves an average improvement of 29.3% and 15.5% in convergence speed compared to the other two DRL-based methods under varying delay fluctuations. Furthermore, the proposed method significantly enhances the normalized throughput compared to conventional Time Division Multiple Access (TDMA) and DOTS protocols.