Semantic Prototype-Guided Sampling for Long-Tailed Generalized Category Discovery

In this work, we address the challenging, real-world data imbalance problem in the context of Generalized Category Discovery (GCD). Here, given a large set of labeled and unlabeled data, the goal is to simultaneously classify known classes and also discover novel classes. Towards this goal, we propose an effective framework, ProtoGuide, which consists of two novel modules built on top of a vision transformer with contrastive learning, namely (i) Text-Enhanced Prototype Generation: where we utilize the text embedding of each labeled class names along with the augmented images for computing semantically meaningful class prototypes; and (ii) Prototype-guided sampling: where we dynamically sample examples guided by the current class prototypes for the minority classes based on the predicted imbalance. These modules along with utilizing the nearest neighbor information for each instance in an unsupervised contrastive manner allow us to develop strong feature representations, thereby enhancing clustering performance. Extensive experiments on multiple benchmarks show that ProtoGuide significantly outperforms the state-of-the-art for this challenging task.