Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification
Duplicate and near-duplicate images can compromise deep learning evaluation through cross-partition leakage. This study investigated duplicate-aware evaluation and reproducibility using 667 public oral photographs representing Caries (219), Mouth Ulcer (265), and Tooth Discoloration (183). Exact and near-duplicate content was examined using cryptographic and perceptual hashing, supplemented by human-reference assessment of 378 image pairs. The automated audit initially identified 580 duplicate-content groups, refined to 564 following human validation. A corrected group-disjoint split (466 training, 101 validation, 100 test images) was used to evaluate ImageNet-pretrained ResNet50, EfficientNet-B3, and ViT-B/16 across three seeds. Performance, calibration, and uncertainty were assessed using a standardized protocol and 10,000 paired duplicate-group bootstrap resamples. ViT-B/16 achieved the highest mean accuracy (92.67% ± 1.15 percentage points) and macro-F1 (0.9194 ± 0.0138), followed by ResNet50 (92.00% ± 2.00; 0.9135 ± 0.0213) and EfficientNet-B3 (83.33% ± 2.08; 0.8263 ± 0.0224). The ViT-B/16–ResNet50 accuracy difference was 0.67 percentage points (95% confidence interval: −3.33 to 4.67), indicating uncertain relative superiority. Exploratory evaluation on 1786 externally annotated caries-positive images yielded mean sensitivities of 87.38%, 74.25%, and 69.65% for EfficientNet-B3, ResNet50, and ViT-B/16, respectively, with substantial seed variability. These findings support human-validated duplicate auditing, group-disjoint partitioning, multi-seed testing, and paired uncertainty analysis. Limited disease coverage, unavailable patient identifiers, and positive-only external evaluation constrain conclusions regarding multiclass generalizability and clinical applicability.
Authors
- Yacoub Altarakemah (ORCID: https://orcid.org/0000-0003-3953-2023)
- A. M. Mutawa (ORCID: https://orcid.org/0000-0002-5707-2692)
- Karthiga Thirupathy
Institutions
- Kuwait University (KW)
Publication Details
- Journal
- Computers
- Published
- 2026-11-13
- DOI
- https://doi.org/10.3390/computers15110830
- Primary Topic
- Dental Radiography and Imaging
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Kuwait Foundation for the Advancement of Sciences