Few-shot Image Recognition with Enhanced Local Features and Semantic Prompt

Compared to traditional large-scale supervised deep learning, few-shot learning aims to utilize limited samples to identify new classes. Recently, Transformer models have been utilized for extracting image features without explicitly modeling spatial locality, instead capturing dependencies at arbitrary positions within the input sequence through self-attention mechanisms. However, this may limit the effective utilization of local structural information between pixels when processing image data. We propose an approach that integrates Convolutional Neural Networks (CNNs) to effectively utilize spatial information, addressing the challenge of traditional large-scale supervised deep learning in identifying new classes with limited samples. By employing convolutional operations with varying kernel sizes, we capture diverse receptive fields, thus preserving spatial structural information through a specifically designed spatial attention mechanism. We seamlessly integrate CNNs into the Transformer model by replacing its input layer with convolutional blocks and introducing pooling between stages to reduce sequence length. Furthermore, we enhance the feature extraction process by incorporating semantic cues through two complementary mechanisms: one involving semantic cue interaction with image patches via self-attention, while the other integrates transformed semantic cues into visual features. This combined approach not only boosts class-specific functionality but also expands the capabilities of image representation, yielding impressive performance even with limited support samples. Extensive experiments conducted on the MiniImageNet dataset validate our approach, achieving an accuracy of 72.92%.