CLIBAD: Zero-Shot Localization for Annotation Error in Object Detection Datasets

Object detection(OD) serves as the cornerstone for building reliable autonomous driving systems. However, as the scale of data continues to expand, errors in annotated data have become increasingly prevalent. Annotation errors not only negatively impact the training process of detection models but also interfere with subsequent workflows such as model evaluation and deployment. Efficiently locating annotation errors in object detection datasets is crucial. Annotation error localization refers to identifying mislabeled annotations and detecting missing annotations in the background. Existing object detection annotation error localization methods suffer from insufficient generalization and weak capability in identifying missing annotations. To address these issues, this paper proposes the CLIBAD method, which consists of two key components: Adaptive Fine-Grained Description (FGAD) and Localization Assistance Module (LAM). FGAD utilizes Large Language Models (LLMs) to generate ffne-grained anomaly descriptions for each category and employs adaptively learned textual templates, thereby enhancing the accuracy and interpretability of detecting incorrect bounding box annotations. LAM utilizing Grounding DINO for preliminary localization and position-enhanced text prompts facilitates more accurate localization of possibly missing annotations. Experiments on multiple datasets demonstrate that CLIBAD achieves performance improvements compared with existing methods.