Boosting Open-Vocabulary Object Detection Performance via Class-Agnostic Pseudo-Labels and MultiModal Hybrid Knowledge

Open-vocabulary object detection (OVD) is a significant task identifying objects from categories not included in the training set. Our comprehensive analysis reveals two main issues with existing OVD models: poor generalization of localization network to novel categories and poor quality of class embedding impacting accuracy. We propose two solutions: Localization Network Enhancement based on Class-Agnostic Pseudo-Labels (LNE-CAPL) and Class Embedding Enhancement based on MultiModal Hybrid Knowledge (CEE-MMHK). These methods can be used offline to significantly improve the existing OVD models without affecting their training and inference efficiency. Extensive experiments confirm their effectiveness and universality. Code and models will soon be open-sourced.