LightFormer: Vision Transformer for Lightweight Based on Cascade Depthwise Convolution and Mixed Attention

In recent years, the complementary advantages of convolutional neural networks (CNNs) and Transformers have been utilized to achieve significant results in image classification tasks. However, the fusion of these two networks can lead to high model complexity and a large number of parameters, which is not conducive to model deployment. To address this issue, we propose a remote sensing image classification method based on a lightweight convolutional Transformer. First, a hybrid attention module is introduced in the shallow network to obtain local–global information of fine target feature maps from different dimensions. This module achieves the mixed interaction of feature information by multiplying different group features with global weights, avoiding the linearity of the model. Then, a lightweight multihead self-attention (MHSA) is introduced in the deep network to enhance the global information dependency of the image. Finally, the feature optimization is performed using concatenation pooling and applied to image classification. Experiments are conducted on the Aerial Image dataset (AID) and the NWPU45 remote sensing image dataset. The experimental results show that the accuracy on these two datasets reaches 96.59% and 95.34%. Furthermore, the proposed method has a parameter size of 1.15 M and a computational cost of 0.5 G, reducing the parameter size and computational cost compared to other image classification methods. Additionally, we further validated the generalization capability of our method on segmentation tasks, achieving good performance on both the Potsdam and LoveDA datasets.