Comparative Accuracy of a Commercial Multimodal Large Language Model Versus Clinicians in Dental Age and Sex Estimation from Panoramic Radiographs: A Benchmark Study

Background/Objectives: Routine dental imaging captures cumulative dentoalveolar and periodontal changes, offering a non-invasive window into both chronological aging and physiological senescence. While task-specific deep learning models have been widely explored in dental radiology, the zero-shot diagnostic performance of commercial Multimodal Large Language Models (MLLMs) in adult populations remains underinvestigated. The aim of this study was to benchmark the zero-shot diagnostic accuracy and repeatability of a commercial MLLM against human clinicians of varying experience levels in estimating adult dental age and biological sex from panoramic radiographs (OPGs), using documented chronological age and biological sex as the ground truth. Methods: From 216 initially screened records, this retrospective single-center study included 127 standardized, anonymized OPGs from patients (74 men, 53 women; mean chronological age 55.5 ± 15.8 years, range 18–90 years) treated at the Clinical Department of Cranio-Maxillofacial Surgery, Military Institute of Medicine—National Research Institute in Warsaw. Each radiograph was independently evaluated across three trials by Gemini 3.1 Pro (Google, Mountain View, CA, USA) and once by two clinicians—a maxillofacial specialist (>10 years of experience) and a resident (<3 years of experience)—all blinded to clinical metadata. Results: Gemini 3.1 Pro demonstrated excellent repeatability across the three trials (ICC(A,1) = 0.926, ICC(A,3) = 0.974; p = 0.156) and moderate agreement with chronological age (ICC(A,1) = 0.604), but systematically overestimated age by a mean of 6.23 years. Mean absolute error (MAE) ranged from 12.91 to 13.03 years across the AI trials. The specialist achieved a significantly lower MAE (10.39 years) than all AI assessments (adjusted p ≤ 0.003), whereas the resident (MAE = 11.19 years) did not differ significantly from the AI (adjusted p = 0.282). In biological sex classification the specialist (66.1%) significantly outperformed all three AI trials (41.7–49.6%), while the resident (62.2%) significantly outperformed the first two AI trials (41.7% and 43.3%), but did not differ significantly from the third trial (49.6%). Across all runs, the AI exhibited significant directional asymmetry (p < 0.001), frequently misclassifying male patients as female. Conclusions: While Gemini 3.1 Pro demonstrated high internal repeatability and age-estimation accuracy comparable to a junior resident, its wide limits of agreement, systematic age overestimation, and poor sex-classification accuracy preclude its autonomous use in clinical or forensic workflows. Future studies incorporating healthy control cohorts and validated clinical indices are required to determine whether the observed age overestimation reflects inherent algorithmic miscalibration or sensitivity to cumulative dentoalveolar pathology in hospitalized patients.

Authors

Institutions

Publication Details

Journal
Journal of Clinical Medicine
Published
2026-11-20
DOI
https://doi.org/10.3390/jcm15229123
Primary Topic
Forensic Anthropology and Bioarchaeology Studies
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Comparative Accuracy of a Commercial Multimodal Large Language Model Versus Clinicians in Dental Age and Sex Estimation from Panoramic Radiographs: A Benchmark Study

Dariusz Jurkiewicz, Maria Sobol, Aldona Chloupek, Kevin Du et al.
6 links
Journal of Clinical Medicine
Forensic Anthropology and Bioarchaeology Studies
article

Comparative Accuracy of a Commercial Multimodal Large Language Model Versus Clinicians in Dental Age and Sex Estimation from Panoramic Radiographs: A Benchmark Study

Dariusz Jurkiewicz, Maria Sobol, Aldona Chloupek, Kevin Du, Dominika Zawadka-Modras
article en

Abstract

Background/Objectives: Routine dental imaging captures cumulative dentoalveolar and periodontal changes, offering a non-invasive window into both chronological aging and physiological senescence. While task-specific deep learning models have been widely explored in dental radiology, the zero-shot diagnostic performance of commercial Multimodal Large Language Models (MLLMs) in adult populations remains underinvestigated. The aim of this study was to benchmark the zero-shot diagnostic accuracy and repeatability of a commercial MLLM against human clinicians of varying experience levels in estimating adult dental age and biological sex from panoramic radiographs (OPGs), using documented chronological age and biological sex as the ground truth. Methods: From 216 initially screened records, this retrospective single-center study included 127 standardized, anonymized OPGs from patients (74 men, 53 women; mean chronological age 55.5 ± 15.8 years, range 18–90 years) treated at the Clinical Department of Cranio-Maxillofacial Surgery, Military Institute of Medicine—National Research Institute in Warsaw. Each radiograph was independently evaluated across three trials by Gemini 3.1 Pro (Google, Mountain View, CA, USA) and once by two clinicians—a maxillofacial specialist (>10 years of experience) and a resident (<3 years of experience)—all blinded to clinical metadata. Results: Gemini 3.1 Pro demonstrated excellent repeatability across the three trials (ICC(A,1) = 0.926, ICC(A,3) = 0.974; p = 0.156) and moderate agreement with chronological age (ICC(A,1) = 0.604), but systematically overestimated age by a mean of 6.23 years. Mean absolute error (MAE) ranged from 12.91 to 13.03 years across the AI trials. The specialist achieved a significantly lower MAE (10.39 years) than all AI assessments (adjusted p ≤ 0.003), whereas the resident (MAE = 11.19 years) did not differ significantly from the AI (adjusted p = 0.282). In biological sex classification the specialist (66.1%) significantly outperformed all three AI trials (41.7–49.6%), while the resident (62.2%) significantly outperformed the first two AI trials (41.7% and 43.3%), but did not differ significantly from the third trial (49.6%). Across all runs, the AI exhibited significant directional asymmetry (p < 0.001), frequently misclassifying male patients as female. Conclusions: While Gemini 3.1 Pro demonstrated high internal repeatability and age-estimation accuracy comparable to a junior resident, its wide limits of agreement, systematic age overestimation, and poor sex-classification accuracy preclude its autonomous use in clinical or forensic workflows. Future studies incorporating healthy control cohorts and validated clinical indices are required to determine whether the observed age overestimation reflects inherent algorithmic miscalibration or sensitivity to cumulative dentoalveolar pathology in hospitalized patients.

Journal of Clinical MedicineVol. 15(22)
Medical University of Warsaw (PL), Wojskowy Instytut Medyczny (PL)
Good health and well-being
Forensic Anthropology and Bioarchaeology Studies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon