Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification

Duplicate and near-duplicate images can compromise deep learning evaluation through cross-partition leakage. This study investigated duplicate-aware evaluation and reproducibility using 667 public oral photographs representing Caries (219), Mouth Ulcer (265), and Tooth Discoloration (183). Exact and near-duplicate content was examined using cryptographic and perceptual hashing, supplemented by human-reference assessment of 378 image pairs. The automated audit initially identified 580 duplicate-content groups, refined to 564 following human validation. A corrected group-disjoint split (466 training, 101 validation, 100 test images) was used to evaluate ImageNet-pretrained ResNet50, EfficientNet-B3, and ViT-B/16 across three seeds. Performance, calibration, and uncertainty were assessed using a standardized protocol and 10,000 paired duplicate-group bootstrap resamples. ViT-B/16 achieved the highest mean accuracy (92.67% ± 1.15 percentage points) and macro-F1 (0.9194 ± 0.0138), followed by ResNet50 (92.00% ± 2.00; 0.9135 ± 0.0213) and EfficientNet-B3 (83.33% ± 2.08; 0.8263 ± 0.0224). The ViT-B/16–ResNet50 accuracy difference was 0.67 percentage points (95% confidence interval: −3.33 to 4.67), indicating uncertain relative superiority. Exploratory evaluation on 1786 externally annotated caries-positive images yielded mean sensitivities of 87.38%, 74.25%, and 69.65% for EfficientNet-B3, ResNet50, and ViT-B/16, respectively, with substantial seed variability. These findings support human-validated duplicate auditing, group-disjoint partitioning, multi-seed testing, and paired uncertainty analysis. Limited disease coverage, unavailable patient identifiers, and positive-only external evaluation constrain conclusions regarding multiclass generalizability and clinical applicability.

Authors

Institutions

Publication Details

Journal
Computers
Published
2026-11-13
DOI
https://doi.org/10.3390/computers15110830
Primary Topic
Dental Radiography and Imaging
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification

Yacoub Altarakemah, A. M. Mutawa, Karthiga Thirupathy
5 links
Computers
Dental Radiography and Imaging
article

Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification

Yacoub Altarakemah, A. M. Mutawa, Karthiga Thirupathy
article en

Abstract

Duplicate and near-duplicate images can compromise deep learning evaluation through cross-partition leakage. This study investigated duplicate-aware evaluation and reproducibility using 667 public oral photographs representing Caries (219), Mouth Ulcer (265), and Tooth Discoloration (183). Exact and near-duplicate content was examined using cryptographic and perceptual hashing, supplemented by human-reference assessment of 378 image pairs. The automated audit initially identified 580 duplicate-content groups, refined to 564 following human validation. A corrected group-disjoint split (466 training, 101 validation, 100 test images) was used to evaluate ImageNet-pretrained ResNet50, EfficientNet-B3, and ViT-B/16 across three seeds. Performance, calibration, and uncertainty were assessed using a standardized protocol and 10,000 paired duplicate-group bootstrap resamples. ViT-B/16 achieved the highest mean accuracy (92.67% ± 1.15 percentage points) and macro-F1 (0.9194 ± 0.0138), followed by ResNet50 (92.00% ± 2.00; 0.9135 ± 0.0213) and EfficientNet-B3 (83.33% ± 2.08; 0.8263 ± 0.0224). The ViT-B/16–ResNet50 accuracy difference was 0.67 percentage points (95% confidence interval: −3.33 to 4.67), indicating uncertain relative superiority. Exploratory evaluation on 1786 externally annotated caries-positive images yielded mean sensitivities of 87.38%, 74.25%, and 69.65% for EfficientNet-B3, ResNet50, and ViT-B/16, respectively, with substantial seed variability. These findings support human-validated duplicate auditing, group-disjoint partitioning, multi-seed testing, and paired uncertainty analysis. Limited disease coverage, unavailable patient identifiers, and positive-only external evaluation constrain conclusions regarding multiclass generalizability and clinical applicability.

ComputersVol. 15(11)
Kuwait University (KW)
Kuwait Foundation for the Advancement of Sciences
Good health and well-being
Dental Radiography and Imaging
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon