Unsupervised Learning-Based Optical–Acoustic Fusion Interest Point Detector for AUV Near-Field Exploration of Hydrothermal Areas

The simultaneous localization and mapping (SLAM) technique provides long-term near-seafloor navigation for autonomous underwater vehicles (AUVs). However, the stability of the interest point detector (IPD) remains challenging in the seafloor environment. This paper proposes an optical–acoustic fusion interest point detector (OAF-IPD) using a monocular camera and forward-looking sonar. Unlike the artificial feature detectors most underwater IPDs adopt, a deep neural network model based on unsupervised interest point detector (UnsuperPoint) was built to reach stronger environmental adaption. First, a feature fusion module based on feature pyramid networks (FPNs) and a depth module were integrated into the system to ensure a uniform distribution of interest points in depth for improved localization accuracy. Second, a self-supervised training procedure was developed to adapt the OAF-IPD for unsupervised training. This procedure included an auto-encoder framework for the sonar data encoder, a ground truth depth generation framework for the depth module, and optical–acoustic mutual supervision for the fuse module training. Third, a non-rigid feature filter was implemented in the camera data encoder to mitigate the interference from non-rigid structural objects, such as smoke emitted from active vents in hydrothermal areas. Evaluations were conducted using open-source datasets as well as a dataset captured by the research team of this paper from pool experiments to prove the robustness and accuracy of the newly proposed method.