A Multimodal Framework for Few-Shot Image Classification Based on Deep Learning

With the continuous evolution of artificial intelligence and big data technologies, deep learning has achieved remarkable progress in the field of computer vision. However, the heavy reliance of conventional deep learning methods on large-scale labeled datasets poses significant challenges in real-world applications. Few-shot learning aims to achieve accurate classification with only a small number of labeled samples, thereby enhancing the model’s generalization ability. To address the limitations of existing methods in feature extraction and the insufficient utilization of multimodal information, this study proposes a novel few-shot classification algorithm based on multi-scale multimodal fusion. For the image modality, a deeper ResNet-18 is adopted as the backbone network, enhanced by group convolutions and hierarchical residual connections. In addition, the EMA attention mechanism is embedded to improve the model’s focus on salient regions and semantic channels. For the text modality, a CLIP text encoder is used to extract semantic prior information. A text-guided semantic alignment module is also designed, where a multi-layer perceptron is employed to generate attention weights, thereby enhancing the fusion between visual and textual information. Furthermore, the method jointly optimizes the model using CORAL alignment loss and cross-entropy loss to achieve both multimodal distribution alignment and classification accuracy improvement. Experimental results demonstrate that the proposed method achieves excellent performance in multimodal few-shot classification tasks.