Text-Guided Dual Feature Enhancement: A Training-Free Paradigm for Few-Shot Learning with CLIP
The key challenge in few-shot image classification lies in effectively utilizing multi-modal prior knowledge to overcome the representation bottleneck caused by limited samples. This paper proposes a training-free Text-guided Dual Feature Enhancement (TDFE) method based on the CLIP framework, which improves classification performance through spatial-channel co-optimization: 1) A text-guided spatial attention module is developed to dynamically enhance key region features using semantic descriptions generated by LLMs; 2) A cross-modal channel fusion mechanism is designed to jointly analyze textual and visual channel characteristics for strengthening discriminative dimensions; 3) A dual-path classifier is introduced to integrate enhanced features with original CLIP zero-shot results. Experiments demonstrate that with ResNet50 as the visual encoder, our method achieves an average accuracy of 71.6% on 16-shot tasks, outperforming TIP-Adapter by 1.3%. Visualization results confirm the model's precise attention focus on text-relevant regions. This study provides an efficient solution for multi-modal learning in data-scarce scenarios.
Authors
- Dongcheng Li (ORCID: https://orcid.org/0000-0001-9598-0651)
- Ruixuan Wang (ORCID: https://orcid.org/0000-0003-2575-0142)
Institutions
- Sun Yat-sen University (CN)
Publication Details
- Published
- 2025-04-11
- DOI
- https://doi.org/10.1109/ainit65432.2025.11035349
- Primary Topic
- Domain Adaptation and Few-Shot Learning
- Type
- article
- Field-Weighted Citation Impact
- 0.00