CDRFormer: CNN and dimensional reciprocal attention mixing transformer network for remote sensing images semantic segmentation

Semantic segmentation of remote sensing images is a fundamental task in geospatial research. Due to the complex ground objects, rich feature details, large intraclass variance, and small interclass variance in remote sensing images, deep learning semantic segmentation methods are usually required to have excellent feature learning representation ability. The widely used methods based on convolutional neural networks (CNNs) and Transformer hybrid architectures are effective but still have shortcomings: CNNs are good at capturing local details but perform poorly in modeling long-range dependencies. Transformers rely on multi-head self-attention mechanisms to extract global contextual information, but this often leads to high computational complexity. Therefore, in this paper, we propose a network based on CNN and Dimensional Reciprocal Attention Mixing Transformer (CDRFormer), a novel encoder-decoder structured network for remote sensing image semantic segmentation. Specifically, for the features extracted by the lightweight ResNet18 encoder, we introduce a Transformer block named Dimensional Reciprocal Attention Mixing Transformer (DRAMiT) to enhance the feature learning ability. DRAMiT includes a two-dimensional self-attention (SA) mixing module. This module runs spatial and channel self-attention mechanisms in parallel with different numbers of heads and mixes them, ultimately capturing both local and global dependencies at a lower computational cost. We tested our model on the ISPRS Vaihingen dataset. Experimental results show that our proposed CDRFormer performs excellently on the ISPRS Vaihingen dataset.