Toward Communication Optimization for Future Underwater Networking: A Survey of Reinforcement Learning-Based Approaches

As an essential part of the 6G sea-land-air integrated network, underwater networking has attracted increasing attention and has been widely studied. The key for improving its performance is the communication optimization based on data rate, throughput, latency, reliability, spectrum utilization, and other factors impacting on the quality of service (QoS). However, the poor underwater communication environment makes it difficult to improve the communication quality of underwater networking and brings many challenges to the design of optimization schemes. In the face of complex and unknown dynamic underwater environment, the optimization schemes need to have a higher level of adaptability and intelligence, so as to carry out autonomous decision-making and multi-objective optimization under different conditions. To meet the above challenges and needs, reinforcement learning (RL) is widely used to obtain the optimal strategy for underwater communication. Nevertheless, there is still a lack of comprehensive reviews on using RL to optimize underwater communication networking. Therefore, this survey comprehensively investigates the application of RL in underwater networking to guide the optimization of underwater communication in the future and bridge this gap. Specifically, we provide an overview of RL usage processes and tools and detail its various applications in underwater communication networking, including spectrum resource allocation and development, throughput improvement and delay reduction, reliability improvement, energy saving, and energy efficiency optimization, data sensing and processing, and intelligent cluster networking. Based on the review, we further analyze the open challenges and research directions of RL-enabled underwater communication networking in the future.