Study on 3D Path Planning of AUV Based on the Reinforcement Learning Method

Autonomous Underwater Vehicles (AUVs) face significant challenges in 3D path planning due to dynamic environmental disturbances, high-dimensional search spaces, and the need for physically feasible trajectories under thruster constraints. While traditional sampling-based and optimization-driven methods struggle with computational complexity and kinematic oversimplification, existing reinforcement learning (RL) approaches often neglect hydrodynamic dynamics, resulting in jerky or unstable paths. This paper proposes a Deep Deterministic Policy Gradient (DDPG)-based framework that integrates rigorous six degrees of freedom (6-DOF) dynamic modeling with adaptive RL to address these limitations. The framework incorporates a novel reward function balancing collision avoidance, trajectory smoothness, and energy efficiency, alongside a kinematic/dynamic model that accounts for hydrodynamic forces, buoyancy, and environmental disturbances (e.g., ocean currents). Key innovations include: 1) A 6-DOF dynamic-aware RL architecture ensuring traj ectories adhere to AUV actuation limits. 2) A progressive training strategy combining curriculum learning and prioritized experience replies to enhance robustness in cluttered 3D environments. 3) A smoothness penalty metric to minimize abrupt directional changes, improving hydrodynamic feasibility. Extensive simulations validate the framework's ability to harmonize real-time adaptability with physical realizability, offering a robust solution for complex underwater applications such as deep-sea exploration and infrastructure inspection.