Enhancing AUV 3D Obstacle Avoidance: A Novel Approach with Self-Supervised Network for Fusion of Forward-Looking Camera and Sonar Data

Autonomous underwater vehicles (AUVs) are typically programmed to follow routes based on predefined waypoints and depth profiles. However, in complex and unpredictable environments-such as coral reefs, offshore structures, or ship-wrecks-AUVs can encounter unexpected obstacles that pose risks to both the vehicle and its surroundings. To navigate and avoid unexpected obstacles, AUVs operating in such environments are often equipped with forward-looking sonars (FLS). However, standard FLS sensors are typically limited in resolution and can only provide 2D information on bearing and range, restricting their effectiveness in facilitating navigation in complex environments. Vision cameras, on the other hand, offer high-resolution data with bearing and elevation information, but when using a single-camera setup, they cannot reliably provide distance information. This study introduces a comprehensive framework for the fusion of forward-looking camera (FLC) and FLS data, using a projection of FLS data into the FLC frame and incorporating data from a trained self-supervised network.