Coalition Game-Based Reinforcement Learning in Underwater Acoustic Networks

Underwater Acoustic (UWA) communication plays a pivotal role in maritime operations, yet it faces persistent challenges such as limited bandwidth, high interference, and dynamic environmental conditions. The purpose of this work is to address the limitations of existing resource allocation strategies that rely on static optimization or centralized control, which lack scalability and adaptability in large, dynamic UWA networks. Hence, we propose a novel and scalable framework, herein referred to as Cooperative Coalition Optimization based on Reinforcement Learning for Underwater network (U-CoCRL) that integrates coalition game theory with Q-learning (QL) to optimize resource allocation in UWA networks. The proposed approach dynamically forms coalitions based on real-time conditions through K-means clustering, ensures fair payoff distribution via the Shapley value and continuously adapts to changing environmental conditions and interference levels via Reinforcement learning (RL). Simulation results demonstrate that the U-CoCRL method outperforms baseline algorithms across all interference levels. It achieves up to 18.1% higher global utility and 33.9% better performance indices under high interference, 13.6% and 33.9% improvements under medium interference, and 6.1% and 27.7% gains under low interference, respectively. Additionally, tests with up to 20 users validate the framework’s scalability and robustness under increased network size. These results highlight the potential of combining cooperative game theory with QL and demonstrate the robustness of the method in optimizing resource allocation and fairness in dynamic underwater communication environments.