Evaluating the Robustness and Generalizability of D-LeDe Using Multiple Automotive Datasets

Abstract Data leakage, where semantically or visually similar samples exist across training and test splits, continues to threaten the reliability of object detection benchmarks. The D-LeDe method was recently proposed as a statistical technique for detecting data leakage, showing promise in initial applications. This paper extends the D-LeDe method to evaluate its robustness and generalizability through a focused experimental study. We revisit the KITTI dataset which was previously identified as leakage-prone and introduce a new application of D-LeDe on SODA10M. The core investigation centers on whether visually similar images, measured using perceptual hashing, are the primary cause of data leakage indications captured by D-LeDe. To this end, we progressively remove visually similar image pairs from the test sets of both datasets and observe changes in the Relative Increase Rate, the key decision metric of D-LeDe. Results show that even after removing about 50% of similar images from the KITTI test set, D-LeDe continues to detect data leakage, suggesting other forms of redundancy or latent factors causing the leakage. In contrast, SODA10M consistently remains leakage-free across all levels of image pair removal. These findings reinforce the reliability of D-LeDe in detecting non-obvious forms of data leakage, underscoring its applicability as a diagnostic tool for dataset integrity in object detection workflows.