Reinforcement Learning-Driven Autonomous Path Planning for Unmanned Surface Vehicles: Current Status, Challenges, and Future Prospects

The continuous advancement of autonomy and intelligence in marine shipping has made the safe and efficient navigation of unmanned surface vehicles in complex waters a major research focus. As a key link of the autonomous decision-making system for unmanned surface vehicles (USVs), local path planning needs to achieve real-time collision avoidance and motion optimization under dynamic obstacles, multiple rule constraints, and strong environmental uncertainty. In recent years, reinforcement learning has gradually become an important technical route for local path planning of USVs by virtue of its autonomous decision-making ability in high-dimensional continuous state space and adaptability to complex nonlinear problems. Combined with the evolution of the algorithm paradigm and its functional positioning in different water scenarios, this paper systematically reviews the relevant literature by examining the evolution of algorithmic paradigms; focuses on summarizing deep Q-network (DQN), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), and Twin Delayed Deep Deterministic Policy Gradient (TD3), along with the collaborative architectures integrated with traditional planning methods such as A* and Rapidly-exploring Random Tree (RRT); and summarizes the performance characteristics, advantages, and limitations of various methods in typical scenarios. The review shows that the main bottlenecks of current research include insufficient reward mechanism design, low sample utilization efficiency, difficulty in transferring from simulation to real ships, and insufficient safety and trustworthiness verification. This paper looks forward to the future development trends from the two directions of data fusion and security enhancement in order to provide reference for related research.