Depth-guided selective cross-modal fusion network for deep-sea polymetallic nodule segmentation

Abstract Accurate segmentation of deep-sea polymetallic nodules is essential for deep-sea mineral resource measurement and assessment. However, existing methods predominantly rely on RGB images, and their performance is often compromised by complex seabed backgrounds, substantial scale variations, and blurred boundaries. To address these challenges, we propose a depth-guided selective cross-modal fusion network (DSCFNet), which uses monocularly estimated depth as a structural prior to complement RGB appearance information. First, a Frequency-Enhanced Modality Interaction (FEMI) module is developed to enhance cross-modal feature representation and suppress redundant responses in complex backgrounds. Second, a Semantic-Boundary Gated Injection (SBGI) module is designed to selectively incorporate reliable complementary cues while reducing interference from depth noise. Finally, a Multi-Scale Depth Feature Refinement (MSDFR) module is constructed to strengthen the representation of nodules with substantial scale variations. Experimental results on a self-constructed multimodal deep-sea polymetallic nodule dataset demonstrate that the proposed DSCFNet outperforms representative semantic segmentation methods, achieving an mIoU of 92.73% and an F1-score of 94.49%. The proposed method improves segmentation accuracy in complex seabed environments, providing a reliable solution for image-based measurement and assessment of deep-sea polymetallic nodule resources.