Dynamic Topic-Based Hierarchical Prompt Learning for Multi-Label Image Classification

Flat label supervision often constrains multi-label image classification, as it struggles to fully capture inherent label dependencies. It provides limited guidance to the hierarchical features that naturally emerge in Vision Transformers. To address this structural misalignment, we propose Dynamic Topic-based Hierarchical Prompt Learning (DyT-HPL). Instead of relying on predefined and fixed label graphs, DyT-HPL utilizes offline hierarchical clustering to construct multi-granularity semantic priors, from which hierarchical prompts are dynamically retrieved. A frozen visual query branch generates stable semantic queries, which are then used to retrieve discrete prompts from constructed coarse-mid-fine prompt pools. These hierarchical prompts are adaptively injected into different network depths, ensuring that semantic guidance with different abstraction levels is introduced at the most suitable architectural stages. To maintain stable routing and prevent prompt mode collapse, we jointly optimize the architecture with asymmetric classification, surrogate matching, and intra-pool diversity losses. This tripartite design promotes a diverse prompt space and isolates routing updates from final predictions. Comprehensive experiments on MS-COCO, NUS-WIDE, and Corel5k demonstrate that DyT-HPL achieves consistent and favorable performance across diverse settings, highlighting the value of hierarchical semantic guidance with different abstraction levels.