Cross Attention Fusion Transformer Network for Urban Remote Sensing Image Segmentation

Urban remote sensing image segmentation algorithms have advanced significantly, but they still encounter challenges such as large scale variations, intra-class heterogeneity, inter-class homogeneity, and unbalanced foreground and background regions. In this paper, we propose a multi-scale feature mapping fusion based on transformer and CNN network for remote sensing image segmentation, called Cross Attention Fusion Transformer Network (CAFTNet). We adopt Swin Transformer-Base as an encoder to fuse the encoded features and decoded features by developing a cross-attention feature mapping fusion module to align the feature information for better modeling representation. In the decoder, we design an efficient convolution module to decode the fused features. We hope to obtain better segmentation results by unifying the spatial information of multi-scale features while reducing the semantic differences. The proposed method is conducted on two urban remote sensing datasets, including ISPRS Vaihingen and ISPRS Potsdam. Specifically, our method achieves 87.8% and 85.1 % mIoU on ISPRS Vaihingen and Potsdam datasets, respectively.