Model-Based AUV Path Planning Using Curriculum Learning and Deep Reinforcement Learning on a Simplified Electronic Navigation Chart

Deep Reinforcement Learning (DRL)-based algorithms have demonstrated substantial effectiveness in tackling complex control problems for autonomous underwater vehicles (AUVs). This paper attempts to evaluate reinforcement learning (RL)-based methods for AUV trajectory planning by incorporating a model of a vehicle’s full motion. In this study, the agent (AUV) is assumed to have no prior knowledge of the environment in which it navigates. Instead, it only receives inputs from navigation sensors and a simulated sonar. Additionally, in the article, a reward function is proposed and described, along with its optimization process, to elicit the desired behaviors in the underwater vehicle. The models are trained and tested on simplified electronic navigation chart (ENC) maps, followed by a comparative analysis against five effective classical methods for trajectory planning. The proposed solution enables efficient, collision-free route planning for the autonomous underwater vehicle, taking its motion dynamics into account to reach the designated target successfully.