Improving transparency and adaptability of undersea object recognition systems

Side-scan sonars are commonly deployed on autonomous underwater vehicles for seafloor imaging and search applications. In recent years, deep neural networks have become a common solution for object recognition problems with these systems, achieving impressive performance. However, adaptability and interpretability of these systems remain a challenge. The models used in these systems are typically tuned for recognition of specific categories of objects and, due to the labor required to create segmentation mask labels, are often limited to bounding box prediction. Meta's Segment Anything Model (SAM), released in 2023, provides a solution to add instance segmentation capabilities to existing object recognition systems. SAM may be fine-tuned on side-scan sonar data to predict separate object and acoustic shadow masks, allowing for automatic and robust measurement of undersea objects. In addition, this fine-tuned variant of SAM may then be integrated with existing object recognition algorithms tuned for side-scan sonar image analysis to transform them into instance segmentation algorithms. The end result is a highly descriptive recognition system with customizable reporting behavior based on the dimensions of the object(s) of interest.