Research on Cooperative Hunting Strategy of Unmanned Surface Vehicle Based on Deep Reinforcement Learning

Unmanned Surface Vehicle (USV) swarm will play a key role in the future maritime operations, and the cooperative encirclement of USV is one of the important areas, and the key lies in the cooperative action to form the surrounding situation of the target. However, the traditional cluster encirclement algorithm is limited by the uncertainty of the environment, and its generalization ability is low. Especially, the encirclement of escaping targets involves the fields of game theory, optimal control and so on. These traditional algorithms highly rely on the accurate mathematical modeling of the environmen. The construction of encirclement strategy is not only complex, but also very expensive. In addition, with the increase of the number of research objects and the transformation from single agent system to multi-agent system, the mathematical model of the system will become more and more complex, and it is very difficult to analyze and solve the optimal strategy. As long as the environment and task are changed, the rules of the encirclement strategy for the original task will not be able to adapt to the encirclement task under the new conditions, which are the limitations of the traditional algorithm. In order to improve this problem and achieve better performance in the cooperative scene, this paper uses the deep reinforcement learning algorithm to study the cooperative hunting problem of USV group in the obstacle constrained environment, especially the compound reward function with the guidance and obstacle avoidance characteristics, which helps to improve the tracking performance of USV.