TASA: Text-Anchored State–Space Alignment for Long-Tailed Image Classification
Long-tailed image classification remains challenging for vision-language models. Head classes dominate training while tail classes are underrepresented and noisy, and short prompts with weak text supervision further amplify head bias. This paper presents TASA, an end-to-end framework that stabilizes textual supervision and enhances cross-modal fusion. A Semantic Distribution Modulation (SDM) module constructs class-specific text prototypes by cosine-weighted fusion of multiple LLM-generated descriptions with a canonical template, providing stable and diverse semantic anchors without training text parameters. Dual-Space Cross-Modal Fusion (DCF) module incorporates selective-scan state-space blocks into both image and text branches, enabling bidirectional conditioning and efficient feature fusion through a lightweight multilayer perceptron. Together with a margin-aware alignment loss, TASA aligns images with class prototypes for classification without requiring paired image-text data or per-class prompt tuning. Experiments on CIFAR-10/100-LT, ImageNet-LT, and Places-LT demonstrate consistent improvements across many-, medium-, and few-shot groups. Ablation studies confirm that DCF yields the largest single-module gain, while SDM and DCF combined provide the most robust and balanced performance. These results highlight the effectiveness of integrating text-driven prototypes with state-space fusion for long-tailed classification.
Authors
- Tingfang Jia (ORCID: https://orcid.org/0009-0008-5723-6184)
- Long Li (ORCID: https://orcid.org/0009-0001-6622-6489)
- Zhaoyang Zhang (ORCID: https://orcid.org/0000-0003-2287-2486)
- Huize Cheng (ORCID: https://orcid.org/0009-0009-3396-8468)
- Hongsheng Yue (ORCID: https://orcid.org/0009-0007-0570-3134)
- Yongfeng Bu (ORCID: https://orcid.org/0009-0006-8898-4308)
Institutions
- Chang'an University (CN)
Publication Details
- Journal
- Journal of Imaging
- Published
- 2025-11-13
- DOI
- https://doi.org/10.3390/jimaging11110410
- Primary Topic
- Domain Adaptation and Few-Shot Learning
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Chang'an University