CNN‐transformer dual branch collaborative model for semantic segmentation of high‐resolution remote sensing images

Abstract High‐resolution remote sensing images play an important role in geological surveys, disaster detection, and other fields. However, highly imbalanced ground target classes and easily confused small ground targets pose significant challenges to the semantic segmentation task. We propose IC‐TransUNet, a new dual branch model based on an encoder‐decoder structure that fully exploits the advantages of convolutional neural networks and transformers and considers both detailed information and semantic information capture. Specifically, a lightweight CSwin transformer and InceptionNeXt are used as the dual branch backbone of the model. To further improve the model performance, first, we designed the InceptionNeXt‐CSwin Transformer Fusion Module (ICFM) and Edge Enhancement Module (EEM) to guide the dual branch backbone to obtain features. Second, a detachable Spatial‐channel Attention Fusion Module (SCAFM) is designed to be flexibly inserted into multiple positions of the model. Finally, we designed a decoder with significant performance based on a global local transformer block, SCAFM, and a multilayer perceptron segmentation head. IC‐TransUNet achieved highly competitive performance in experiments on the Vaihingen and Potsdam datasets from the International Society for Photogrammetry and Remote Sensing.