From Simulation to Reality: Deep Reinforcement Learning for Autonomous Underwater Vehicle Docking

Deep Reinforcement Learning (DRL) offers significant potential for real-world robotics applications, yet sim-to-real transfer remains a major challenge. In this work, we propose the use of the Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm to efficiently train a docking policy. We tailor our reward function to perform smooth docking. Additionally, we employ a “training wheels” approach that initially fills the replay buffer with PID controller demonstrations to accelerate learning. The resulting model is applied to visual servoing and docking tasks, utilizing AprilTag markers for localization. Real experiments validate our approach.