Bridging the Sim2Real Gap: Multi‐Fidelity Reinforcement Learning for Autonomous Surface Vehicles

ABSTRACT The deployment of Reinforcement Learning (RL) for Autonomous Surface Vehicles (ASV) faces a significantly slower adoption rate compared to other robotic sectors, primarily due to the safety concerns, operational complexity, and expensive procedures associated with physical maritime testing. Traditional RL training methods typically rely on expensive, high‐fidelity simulations that limit policy iteration speed or lead to significant simulation‐to‐reality gaps during transfer. This work introduces a progressive training framework designed to improve the safety, speed, and cost‐efficiency of autonomous policy development. Policies first learn core navigation behaviors in a fast, low‐fidelity environment before going through a performance‐gated transition to high‐fidelity physics for refinement. A correlation analysis validates effective knowledge transfer between fidelity levels, enabling a 77.1% reduction in training time and minimizing the overall time‐to‐deployment. Within the same computational budget as traditional single‐fidelity training, the framework enables the exploration of approximately 80 distinct policy configurations to identify optimal candidates. To demonstrate operational readiness, the framework was used to develop a set of essential navigation skills, including heading‐restricted waypoint reaching, station‐keeping, and reactive obstacle avoidance, which were validated on a physical ASV in a real harbor environment. Collectively, these skills compose a foundational library for achieving higher‐level behaviors in complex maritime missions. These results confirm that the multi‐fidelity approach effectively bridges the sim‐to‐real gap, delivering robust, collision‐free autonomous behaviors under challenging real‐world maritime conditions reaching Sea State 4.