Inter-class relationship matrix and contrastive distillation for open vocabulary object detection

Knowledge distillation-based open vocabulary object detection (OVOD) algorithm aims to distill the knowledge of Vision-Language Models (VLM) to object detection models. However, existing methods often suffer from overfitting to base classes due to the limited supervision provided by base class data during distillation. To alleviate this problem, this paper introduce a novel approach that incorporates an Inter-class Relationships Matrix (ICRM) and Contrastive Distillation (CD). The ICRM captures the relationships between different categories in the VLM embedding space, while CD leverages positive and negative sample information to enhance feature discriminability. By aligning the ICRM of region embeddings with that of category embeddings and employing CD, our method effectively transfers knowledge from VLM to object detection models. Experiment results on the mainly used datasets of OVOD show that our method effective improves the detection of novel categories and maintaining competitive performance on base categories.