Multi-Level Contextual Prototype Modulation for Compositional Zero-Shot Learning
Compositional Zero-Shot Learning (CZSL) aims to recognize unseen attribute-object compositions by leveraging prior knowledge of known primitives. However, real-world visual features of attributes and objects are often entangled, causing distribution shifts between seen and unseen combinations. Existing methods often ignore intrinsic variations and interactions among primitives, leading to poor feature discrimination and biased predictions. To address these challenges, we propose Multi-level Contextual Prototype Modulation (MCPM), a transformer-based framework with a hierarchical structure that effectively integrates attributes and objects to generate richer visual embeddings. At the feature level, we apply contrastive learning to improve discriminability across compositional tasks. At the prototype level, a subclass-driven modulator captures fine-grained attribute-object interactions, enabling better adaptation to long-tail distributions. Additionally, we introduce a Minority Attribute Enhancement (MAE) strategy that synthesizes virtual samples by mixing attribute classes, further mitigating data imbalance. Experiments on four benchmark datasets (MIT-States, C-GQA, UT-Zappos, and VAW-CZSL) show that MCPM brings significant performance improvements, verifying its effectiveness in complex composition scenes.
Authors
- Jungong Han (ORCID: https://orcid.org/0000-0003-4361-956X)
- Xinshuo Wang
- Ling Shao (ORCID: https://orcid.org/0000-0002-8264-6117)
- Yang Liu (ORCID: https://orcid.org/0000-0001-5917-8653)
- Xinbo Gao (ORCID: https://orcid.org/0000-0003-1443-0776)
Institutions
- Xidian University (CN)
- University of Chinese Academy of Sciences (CN)
- University of Sheffield (GB)
Publication Details
- Journal
- IEEE Transactions on Image Processing
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1109/tip.2025.3592560
- Primary Topic
- Geophysical Methods and Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities