Assessing the reliability of artificial intelligence tools for laypersons seeking basic orthodontic knowledge: focus on retainers, mini-implants, and lateral cephalograms

The rapid emergence of artificial intelligence (AI)-based conversational models has significantly influenced medical and dental education. However, the reliability and accuracy of these systems in orthodontic knowledge remain uncertain. This study aimed to comparatively evaluate the reliability, accuracy, completeness, and clinical applicability of responses generated by ChatGPT, Gemini, and Microsoft Copilot in selected orthodontic domains including retainers, mini-implants, and lateral cephalograms. A cross-sectional comparative study was conducted using 61 orthodontic questions derived from Frequently asked questions from google search bar. Free versions of three AI chatbots ChatGPT (GPT-4o), Gemini (1.5 Flash), and Microsoft Copilot were assessed using identical prompts for layperson target audience under standardized conditions between May 10 and May 15, 2026. Questions were categorized into retainers, mini-implants, and lateral cephalograms. Responses were independently evaluated by five board-certified orthodontists using a validated 5-point rubric (1 = Completely Inaccurate/Poor to 5 = Highly Accurate/Excellent). Inter-examiner reliability was assessed using a two-way random-effects, single-measures intraclass correlation coefficient (ICC (2,1)). Because data across the three AI platforms were paired per question, non-parametric paired comparisons were conducted using the Friedman test, followed by post-hoc pairwise Wilcoxon signed-rank tests with Bonferroni correction (a = 0.017). Effect sizes were calculated using Kendall’s W. Significant differences in overall performance were observed among the AI systems (χ² (2) = 48.32, p < 0.001, Kendall’s W = 0.396). ChatGPT achieved the highest overall mean score (4.31, 95% CI [4.20,4.42]), followed by Gemini (3.94, 95% CI[3.82,4.06]) and Copilot (3.67, 95% CI [3.53,3.81]). Post-hoc testing confirmed that ChatGPT significantly outperformed both Gemini ( p < 0.001) and Copilot ( p < 0.001). Cephalometric queries yielded the highest consistency across models, whereas mini-implant biomechanics exhibited the highest variability. Inter-examiner agreement was excellent (ICC (2,1) = 0.87,95%CI [0.81,0.92]). AI-based conversational models demonstrate promising potential as supplementary educational tools in orthodontics; however, variability in scientific reliability and clinical consistency persists. Careful professional supervision remains essential when utilizing AI-generated orthodontic information in educational or clinical settings. Limitations regarding dynamic model updates and evaluator subjectivity highlight that AI outputs should complement, rather than replace, professional clinical consultations.

Authors

Institutions

Publication Details

Journal
BMC Oral Health
Published
2026-11-14
DOI
https://doi.org/10.1186/s12903-026-10439-7
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Assessing the reliability of artificial intelligence tools for laypersons seeking basic orthodontic knowledge: focus on retainers, mini-implants, and lateral cephalograms

Bushra Anis, Hiba Tabassum, Laraib Nasir, Anam Sattar et al.
BMC Oral Health
Artificial Intelligence in Healthcare and Education
article

Assessing the reliability of artificial intelligence tools for laypersons seeking basic orthodontic knowledge: focus on retainers, mini-implants, and lateral cephalograms

Bushra Anis, Hiba Tabassum, Laraib Nasir, Anam Sattar, Sadia Rizwan, Imtiaz Ahmed
article en

Abstract

The rapid emergence of artificial intelligence (AI)-based conversational models has significantly influenced medical and dental education. However, the reliability and accuracy of these systems in orthodontic knowledge remain uncertain. This study aimed to comparatively evaluate the reliability, accuracy, completeness, and clinical applicability of responses generated by ChatGPT, Gemini, and Microsoft Copilot in selected orthodontic domains including retainers, mini-implants, and lateral cephalograms. A cross-sectional comparative study was conducted using 61 orthodontic questions derived from Frequently asked questions from google search bar. Free versions of three AI chatbots ChatGPT (GPT-4o), Gemini (1.5 Flash), and Microsoft Copilot were assessed using identical prompts for layperson target audience under standardized conditions between May 10 and May 15, 2026. Questions were categorized into retainers, mini-implants, and lateral cephalograms. Responses were independently evaluated by five board-certified orthodontists using a validated 5-point rubric (1 = Completely Inaccurate/Poor to 5 = Highly Accurate/Excellent). Inter-examiner reliability was assessed using a two-way random-effects, single-measures intraclass correlation coefficient (ICC (2,1)). Because data across the three AI platforms were paired per question, non-parametric paired comparisons were conducted using the Friedman test, followed by post-hoc pairwise Wilcoxon signed-rank tests with Bonferroni correction (a = 0.017). Effect sizes were calculated using Kendall’s W. Significant differences in overall performance were observed among the AI systems (χ² (2) = 48.32, p < 0.001, Kendall’s W = 0.396). ChatGPT achieved the highest overall mean score (4.31, 95% CI [4.20,4.42]), followed by Gemini (3.94, 95% CI[3.82,4.06]) and Copilot (3.67, 95% CI [3.53,3.81]). Post-hoc testing confirmed that ChatGPT significantly outperformed both Gemini ( p < 0.001) and Copilot ( p < 0.001). Cephalometric queries yielded the highest consistency across models, whereas mini-implant biomechanics exhibited the highest variability. Inter-examiner agreement was excellent (ICC (2,1) = 0.87,95%CI [0.81,0.92]). AI-based conversational models demonstrate promising potential as supplementary educational tools in orthodontics; however, variability in scientific reliability and clinical consistency persists. Careful professional supervision remains essential when utilizing AI-generated orthodontic information in educational or clinical settings. Limitations regarding dynamic model updates and evaluator subjectivity highlight that AI outputs should complement, rather than replace, professional clinical consultations.

BMC Oral Health
Dow University of Health Sciences (PK)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon