FEL-FRN: fusion ECA long-CLIP feature reconstruction network for few-shot classification

Abstract This paper focuses on the problem of the feature reconstruction network (FRN) facing difficulty in reconstructing the query set in fine-grained classification tasks when the objects in the support set have multiple attributes. To address this problem, we propose a model called FEL-FRN (fusion ECA Long-CLIP feature reconstruction network). First, we use FRN to deeply reconstruct feature maps, replacing the traditional method of using cosine similarity for category average aggregation. Moreover, we introduce the efficient channel attention (ECA) attention mechanism into the FRN to improve the model’s ability to extract key features. Second, by introducing Long-CLIP to assist FRN recognition, the Long-CLIP model with a wide range of image recognition and understanding capabilities is obtained. The model does not require any task-specific fine-tuning data and can be combined with category text prediction. Finally, in each training task, the prediction results of different branches are fused. The Long-CLIP model can effectively compensate for the problem of poor prediction caused by large differences between the reconstructed support images and the poor quality of the reconstructed images, whereas the FRN reconstruction network compensates for the lack of precision in Long-CLIP direct prediction through reconstructed predictions, achieving complementary advantages. The experimental results show that this FEL-FRN method not only achieves good results on CUB-200–2011 and Oxford 102 flowers but also uses 5way5shot as a support set on the car dataset Stanford_Cars and the aircraft dataset FGVC_Aircraft, which have large attribute differences, with accuracies of 96.025% and 81.479%, respectively. The results show that the performance is improved compared with that of the FRN model strategy used alone. https://github.com/feiyeha/FEL-FRN