Reinforcement Learning and Multi-Access Edge Computing for 6G-Based Underwater Wireless Networks

6G networks are envisioned to dramatically enhance the connectivity landscape by integrating communication across ground, air, and sea environments. In the aquatic domain, the Internet of Underwater Things (IoUT) represents a global network of intelligent underwater devices designed to capture, interpret, and share data. Although Underwater Acoustic Communications (UAC) has become widespread as a solution for transmitting information, data collection from Underwater Sensor Nodes (USNs) to the surface results in extensive delays and higher energy consumption. Edge communication emerges as a solution to address these challenges. In this approach, Autonomous Underwater Vehicles (AUVs) bring edge computing as close as possible to the source devices. This paper proposes an innovative AUV-based Multi-Access Edge Computing (MEC) system where cluster-heads that collect data from IoUT devices offload their associated computational tasks to local AUVs. These AUVs are strategically positioned to execute tasks either fully locally, partially, or by offloading them entirely to a more resource-equipped AUV (AUV MEC). We achieve this by jointly optimizing the task offloading strategy, resource allocation, and the trajectories of the AUVs. We formulate a non-convex optimization problem to minimize the weighted sum of service delays for all local AUVs and their energy consumption. To address the NP-hard nature of this problem, we employ a deep reinforcement learning algorithm, Deep Deterministic Policy Gradient (DDPG), to solve it. Extensive simulations have been conducted to evaluate the effectiveness of our proposed communication system. The results show that our proposed algorithm outperforms the Total Offloading (Offloading), Local Execution (Locally), and Actor-Critic (AC) algorithms.