Self-Attention Based Embedding Adaptation for Few-Shot Image Classification

The feature extraction part in existing few-shot learning models often uses simple four-layer convolutional or residual networks, which are not optimized for few-shot contexts. This can lead to extracted features being influenced by parts of the image unrelated to the labeled class, reducing the efficiency of many few-shot learning algorithms. To address this, we proposed the Self-Attention Based Embedding Adaptation (SAEA) method. SAEA introduces a new convolutional layer called Detail-Focused Convolution (DFConv) for better feature extraction without increasing the number of parameters or inference time. Additionally, an embedding adaption function built upon the self-attention mechanism is designed to eliminate the influence of the background part, ensuring that the extracted features belong completely to the labeled class. SAEA has shown promising results on the fine-grained benchmark CUB, as well as the commonly used benchmarks mini-ImageNet and tiered-ImageNet.