Feature-Optimized Vision Transformer for Image Classification

To solve the deficiency of Vision Transformer in local detail modelling ability, this paper proposes a feature enhancement method based on the space channel collaborative attention mechanism. The method introduces a two-dimensional feature enhancement module into the standard ViT architecture and collaboratively optimizes feature representation on the two key dimensions of space and channel. The method includes a parallel spatial attention branch and a channel attention branch, which capture the spatial location information and channel semantic information of the feature map, respectively, and realize the effective fusion of the two through a cross-dimensional feature interaction mechanism. Experiments employ partial weight updates and only updates the EMA module and MLP. The results show that the method achieves a 2% accuracy increase over the baseline ViT model on the CIFAR-10 dataset and a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{2. 2 \%}$</tex> accuracy increase on the CUB_200_2011 dataset, thereby validating the efficacy of space channel collaborative enhancement for input feature optimization.