Adaptive Q-learning for Path Planning of AUV in Obstacle-Dense Environments

Effective trajectory planning is crucial for Autonomous Underwater Vehicles (AUVs) in executing a variety of functions, such as emergency response and logistical support operations. Recently, the domain of dynamic trajectory planning has emerged as a significant area of scholarly inquiry, allowing AUVs to maneuver without reliance on pre-existing static data. Researchers are harnessing the principles of reinforcement learning (RL), which has gained prominence as a contemporary field of research, to address the complexities inherent in dynamic trajectory planning. In this investigation, dynamical detection and avoiding collisions methodology is proffered to facilitate navigation within an unstructured environment replete with obstacles. Firstly, we implement the Q-learning algorithm, a specialized subset within the realm of RL, to confront the challenges posed by trajectory planning in an obstacle-laden environment. At the outset, the agent undergoes training to ascertain an effective path through the environment, subsequently selecting the shortest route while circumventing collisions with obstacles. Evaluations conducted across three distinct settings are conducted to assess the efficacy of the proposed method against randomly generated starting and destination coordinates. The findings obtained reveal that the offered strategy is superior in terms of path efficiency and the duration of travel required to reach the intended destination while successfully evading collisions. The suggested strategy holds the potential to enhance underwater search and rescue operations in inundated urban locales or regions impacted by calamities. The AUVs are adept at navigating through complex and unfamiliar underwater environments, skillfully navigating around debris and obstacles, thereby enabling swift and effective identification of survivors or assessment of structural integrity.