Semantic-assisted Hierarchical Clustering with Context-Enhanced Prompting Based on Reliable Examples

Abstract Semantic-assisted image clustering aims to partition images into semantically meaningful groups by leveraging textual semantics to bridge the visual-semantic gap. However, inconsistent semantic granularity, cross-modal semantic conflicts between vision and language modalities, and insufficient utilization of semantic supervision limit the accuracy and interpretability of existing methods. To address these challenges, we propose a novel framework that constructs semantically consistent cluster label space and achieves label assignment with explicit semantic reasoning. First, the cluster label space with uniform granularity and non-overlapping semantics is constructed via hierarchical clustering on example-level semantic labels. Then we propose a context-enhanced semantic arbitration mechanism to perform robust and interpretable cluster assignment. Extensive experiments on multiple public benchmarks demonstrate that our method achieves state-of-the-art performance with significant improvements across all evaluation metrics.