Machine Learning-Enabled Trajectory Optimization for AUV Relaying in Underwater Communications

Underwater communication (UWC) faces significant challenges due to signal attenuation, multipath effects, and complex underwater topography. Deploying an Autonomous Underwater Vehicle (AUV) as a relay node can improve link reliability; however, optimal positioning and trajectory planning remain inadequately explored. Given the dynamic nature of transmission loss with AUV movement, determining an efficient trajectory is crucial for enhancing end-to-end signal quality. While Deep Q-Networks (DQNs) have been applied to this task, their performance degrades in large and complex environments, especially due to difficulty in handling a long-term trajectory optimization. To overcome this, we propose a multi-staged DQN framework that divides the overall path into shorter segments, applying individual DQNs sequentially to identify optimal local paths. These are then concatenated to form a complete trajectory. A reward threshold mechanism guides exploration toward globally optimal solutions. Simulation results demonstrate that the proposed method outperforms conventional approaches in terms of average cumulative signal-to-noise ratio (SNR) gain, achieving rapid convergence, strong generalization across scenarios, and minimal performance loss in challenging conditions.