Image Classification Based on ViT-X on Small-Scale Datasets

近年来，在图像分类等计算机视觉任务中，Vision Transformer (ViT)展现出了卓越的进展，但ViT网络在建模图像中的局部依赖关系方面常显不足，尤其是在小规模数据集上训练时，可能导致归纳偏置不足的问题。针对该问题，本文提出了一种改进的ViT模型。该模型通过引入功能更强的交叉协方差注意力机制(XCA)，增强对多尺度上下文全局依赖关系的建模能力，同时在保持性能优势的情况下减少参数数量。在此基础上，本文还提出一种新颖的模块(Septh-Wise Convolution，简称SWConv)，进一步增强局部特征提取能力。实验结果表明，本文提出的ViT-X模型在CIFAR10等经典数据集中取得了优异的性能，该模型识别准确率达到95.6%，较原始ViT模型提升了1.8%，显著提高了模型的识别性能。In recent years, Vision Transformers (ViT) have demonstrated remarkable progress in computer vision tasks such as image classification. However, ViT networks often struggle to model local dependencies within images, especially when trained on small-scale datasets, which can lead to insufficient inductive bias. To address this issue, this paper proposes an improved ViT model. The proposed model introduces a more powerful cross-covariance attention mechanism (XCA) to enhance the modeling of multi-scale contextual global dependencies while reducing the number of parameters without compromising performance. Furthermore, a novel module (Septh-Wise Convolution, SWConv) is proposed to further strengthen local feature extraction capabilities. Experimental results show that the proposed ViT-X model achieves outstanding performance on benchmark datasets such as CIFAR10, reaching an accuracy of 95.6%, which is 1.8% higher than the original ViT model, significantly improving the recognition performance of the model.