Path planning of autonomous underwater vehicle for data collection of the internet of everything

Autonomous Underwater Vehicle (AUV) has become an important tool to accomplish various path planning tasks due to its high intelligence and good maneuverability. Aiming at the problem of data collection at underwater Internet of Everything (IoE) nodes, this paper constructs a complex 3D marine environment based on real marine current data, and proposes a path planning algorithm based on reinforcement learning to ensure that the AUV completes the data collection with a short path length. In particular, in order to address the problem of complex path planning tasks, the Parallel Dense neural Network (PDNet) is proposed to improve the performance of the agent by extracting the core features of the input state. In addition, to simplify the reward shaping, we constructed a marine environment with sparse rewards. Sparse rewards can greatly interfere with the agent's exploration and learning. To solve the sparse reward problem, the Hindsight Experience Replay (HER) is introduced, which not only solves the sparse reward problem, but also improves the sampling efficiency and convergence of the algorithm.