ClassifierAlign: Unsupervised multi-label image classification via training-free vision-language classifier alignment.

Existing vision-language models (VLMs) exhibit strong zero-shot generalization, yet remain fundamentally limited in Unsupervised Multi-label Image Classification (UMIC), primarily due to their inadequate multi-object perception and the intrinsic modality gap between vision and language. In particular, global representations tend to entangle multiple concepts, hindering accurate recognition of co-existing objects. To address these challenges, we revisit UMIC from a decomposition-then-classification perspective. Instead of predicting multiple concepts holistically, we first decompose an unlabeled multi-label image into multiple single-concept visual instances, thereby explicitly improving multi-object perception. We then perform classification based on aggregated predictions. Building upon this paradigm, we propose ClassifierAlign, a fully training-free framework that jointly leverages visual foundation models (VFMs) and VLMs. Specifically, we derive a lightweight visual classifier directly in the visual feature space, and semantically align it with CLIP's textual classifier, effectively bridging the vision-language modality gap without updating parameters of the pretrained foundation models. This design further enables seamless integration of powerful self-supervised VFMs (e.g., DINO), yielding robust and transferable representations without requiring spatial alignment. Extensive experiments demonstrate that ClassifierAlign achieves competitive performance with fine-tuning-based methods, while significantly reducing computational overhead and eliminating the need for costly annotated data. The source code for this work can be found at https://github.com/dengxili/ClassifierAlign.

Authors

Institutions

Publication Details

Journal
PubMed
Published
2026-11-14
DOI
https://doi.org/10.1016/j.neunet.2026.109901
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

ClassifierAlign: Unsupervised multi-label image classification via training-free vision-language classifier alignment.

Chengfu yang, Nanqing Liu, Xili Deng, Yaping Zhang et al.
PubMed
Multimodal Machine Learning Applications
article

ClassifierAlign: Unsupervised multi-label image classification via training-free vision-language classifier alignment.

Chengfu yang, Nanqing Liu, Xili Deng, Yaping Zhang, Zhanxuan Hu, Yi Zhao
article en

Abstract

Existing vision-language models (VLMs) exhibit strong zero-shot generalization, yet remain fundamentally limited in Unsupervised Multi-label Image Classification (UMIC), primarily due to their inadequate multi-object perception and the intrinsic modality gap between vision and language. In particular, global representations tend to entangle multiple concepts, hindering accurate recognition of co-existing objects. To address these challenges, we revisit UMIC from a decomposition-then-classification perspective. Instead of predicting multiple concepts holistically, we first decompose an unlabeled multi-label image into multiple single-concept visual instances, thereby explicitly improving multi-object perception. We then perform classification based on aggregated predictions. Building upon this paradigm, we propose ClassifierAlign, a fully training-free framework that jointly leverages visual foundation models (VFMs) and VLMs. Specifically, we derive a lightweight visual classifier directly in the visual feature space, and semantically align it with CLIP's textual classifier, effectively bridging the vision-language modality gap without updating parameters of the pretrained foundation models. This design further enables seamless integration of powerful self-supervised VFMs (e.g., DINO), yielding robust and transferable representations without requiring spatial alignment. Extensive experiments demonstrate that ClassifierAlign achieves competitive performance with fine-tuning-based methods, while significantly reducing computational overhead and eliminating the need for costly annotated data. The source code for this work can be found at https://github.com/dengxili/ClassifierAlign.

PubMedVol. 206(Pt C)
Yunnan Normal University (CN)
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon