DPKD:A USV Target Search Algorithm Combine with DDPG and Prior Knowledge Data

Target searching of Unmanned Surface Vehicles (USVs) has played a significant role in maritime rescue, patrolling, and homeland defense, etc. In this paper, a pre-training-based deep reinforcement learning approach to achieve rapid convergence of the USV target search model. In our methodology, firstly, we pre-train the USV search strategy using supervised learning based on human prior knowledge data, thereby endowing the USV with preliminary search capabilities. Then, we employ the Deep Deterministic Policy Gradient (DDPG) algorithm for modeling the USV search strategy. At this stage, we transfer the pre-trained parameters to the DDPG network for initial parameter adjustment of the model. Finally, we further optimize the policy model through interactive training mechanisms of deep reinforcement learning until convergence is achieved. For validating the effectiveness of the present method, we construct a representative USV target search task environment and collect multiple datasets as pre-training data in this environment. Experimental results demonstrate our algorithm is better than the DDPG algorithm with respect to model convergence speed, while effectively overcoming local optima, thus exhibiting stronger capabilities in target search tasks.