A fine‐grained image classification method based on information interaction

Abstract To enhance the accuracy of fine‐grained image classification and address challenges such as excessive interference factors within the dataset, inadequate extraction of local key features, and insufficient channel semantic association, a dual‐branch information interaction model that integrates convolutional neural networks (CNN) with Vision Transformers is proposed. This model leverages the Vision Transformer branch to extract global features, which are subsequently combined with the CNN branch to further augment the model's capability for local information extraction. In order to enhance the ability of the CNN branch to extract global information and reduce the loss of feature information, a feature enhancement module is added to the CNN branch. Since the Vision Transformer branch directly convolves with the convolution kernel will result in the inability to learn the underlying features of the image, a shallow feature extraction module is proposed, and the CNN and Vision Transformer branches interact with the information of the dual branches through the down‐sampling Down module and the up‐sampling UP module. The accuracy of the improved method on CUB‐200‐2011, Stanford Cars and FGVC‐Aircraft fine‐grained image classification datasets are 95.2%, 97.1% and 96.9%, respectively. The experimental results show that the method has good generalization on different datasets.