Open-vocabulary object detection via Neighboring Region Attention Alignment

The nature of diversity in real-world environments necessitates neural network models to expand from closed category settings to accommodate novel emerging categories. In this paper, we study open-vocabulary object detection (OVD), which facilitates the detection of novel object classes under the supervision of only base annotations and open-vocabulary knowledge. However, we find that the inadequacy of distilled information from the detector head during the alignment process inevitably constrains the performance of recent distillation-based OVD strategies. To this end, we propose Neighboring Region Attention Alignment (NRAA), which performs alignment within the attention mechanism to boost the open-vocabulary inference. Specifically, for a given proposal region, we randomly explore the neighboring boxes to capture the surrounding contextual vocabulary knowledge. Then, a set of regional token features, encompassing both the proposal and neighboring regions, utilize our proposed Neighboring Region Attention (NRA) to extract interaction information. Finally, this information is seamlessly provided to the distillation procedure to assist the alignment between the detector and the pre-trained vision-language models (VLMs). Extensive experiments validate that our proposed model exhibits superior performance on open-vocabulary benchmarks. • Propose the neighboring region attention mechanism for token interaction. • Integrate token interactions into the detection head to enhance contrastive alignment. • The model improves open-vocabulary detection performance across benchmarks.