Integrating deep Q-networks, convolutional neural networks, and artificial potential fields for enhanced search path planning of unmanned surface vessels

Unmanned surface vessels (USVs) have become more prevalent in both military and civilian operations in recent years. This paper addresses the issue of search path planning for USVs. To address this problem, a dynamic probability model of the target is constructed using the Monte Carlo Method and a hidden Markov model (HMM). The ship sonar detection model is constructed, and the cumulative detection probability (CDP) is used as an index of searching efficiency. Based on the target model, a path-planning method based on deep Q-network (DQN) is proposed, and a convolutional neural network (CNN) is introduced to adapt the search model on this basis. An artificial potential field (APF) method is proposed to enhance the sparse reward function in the search environment. Finally, the effectiveness of the model and algorithm developed in this paper is verified using simulation experiments and compared to the traditional path planning algorithm, demonstrating the superiority of the algorithm proposed in this paper. • Constructed a dynamic target probability model using Monte Carlo and HMM for USV path planning in on-call search scenarios. • Proposed a CNN-based path planning method to adapt the search model to changing target probability distributions. • Enhanced sparse reward functions with artificial potential fields to accelerate convergence in USV search environments.