Vision–Language Models Empowered Nighttime Object Detection With Consistency Sampler and Hallucination Feature Generator
Current object detectors often suffer performance degradation when applied to cross-domain scenarios, particularly under challenging visual conditions such as nighttime scenes. This is primarily due to the I3 problems: Inadequate sampling of instance-level features, Indistinguishable feature representation across domains and Inaccurate generation for identical category participation. To address these challenges, we propose a domain-adaptive detection framework that enables robust generalization across different visual domains without introducing any additional inference overhead. The framework comprises three key components. Specifically, the centerness-category consistency sampler alleviates inadequate sampling by selecting representative instance-level features, while the paired centerness consistency loss enforces alignment between classification and localization. Second, VLM-based orthogonality enhancement leverages frozen vision-language encoders with an orthogonal projection loss to improve cross-domain feature distinguishability. Third, hallucination feature generator synthesizes robust instance-level features for missing categories, ensuring balanced category participation across domains. Extensive experiments on multiple datasets covering various domain adaptation and generalization settings demonstrate that our method consistently outperforms state-of-the-art detectors, achieving up to 5.5 mAP improvement, with particularly strong gains in nighttime adaptation.
Authors
- Junjie Ke (ORCID: https://orcid.org/0000-0001-8512-3744)
- Lihuo He (ORCID: https://orcid.org/0000-0002-0555-3574)
- Qi Wang (ORCID: https://orcid.org/0000-0002-7028-4956)
- Jie Li (ORCID: https://orcid.org/0000-0001-7950-4233)
- Xinbo Gao (ORCID: https://orcid.org/0000-0002-7985-0037)
- Kai Zhou (ORCID: https://orcid.org/0000-0001-9859-1758)
- Zhenghao Wang (ORCID: https://orcid.org/0000-0003-2676-7683)
Institutions
- Xidian University (CN)
- Northwestern Polytechnical University (CN)
- Chinese University of Hong Kong, Shenzhen (CN)
- Tsinghua University (CN)
Publication Details
- Journal
- IEEE Transactions on Image Processing
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1109/tip.2025.3641316
- Primary Topic
- Advanced Neural Network Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00