A Multimodal Cross-Domain Segmentation Network of Remote Sensing Imagery With Multilevel Deep Cross-Fusion and Adversarial Domain Adaptation

Deep learning techniques have recently achieved significant success in semantic segmentation for Earth observation tasks using single-modality data within a specific region. However, such optimal conditions are seldom found in real-world applications, leading to subpar performance and limited generalization of existing models when faced with multimodal cross-domain scenarios. To tackle this issue, we introduce a novel segmentation network for remote sensing imagery (RSI) called MC-Seg, which incorporates multilevel deep cross-fusion and adversarial domain adaptation. The MC-Seg framework begins by performing multilevel deep cross-fusion of multimodal data through a complete feature extraction module and three fusion levels: feature, modal, and layer. This approach ensures the thorough utilization of features from various modalities. Following this, the framework integrates a local-global adversarial domain adaptation module to minimize the domain gap, effectively aligning the source and target domains despite the differences in RSI data representations. Experimental validation on the C2Seg dataset shows that our proposed method outperforms existing state-of-the-art techniques.