Assessing the reliability of artificial intelligence tools for laypersons seeking basic orthodontic knowledge: focus on retainers, mini-implants, and lateral cephalograms
The rapid emergence of artificial intelligence (AI)-based conversational models has significantly influenced medical and dental education. However, the reliability and accuracy of these systems in orthodontic knowledge remain uncertain. This study aimed to comparatively evaluate the reliability, accuracy, completeness, and clinical applicability of responses generated by ChatGPT, Gemini, and Microsoft Copilot in selected orthodontic domains including retainers, mini-implants, and lateral cephalograms. A cross-sectional comparative study was conducted using 61 orthodontic questions derived from Frequently asked questions from google search bar. Free versions of three AI chatbots ChatGPT (GPT-4o), Gemini (1.5 Flash), and Microsoft Copilot were assessed using identical prompts for layperson target audience under standardized conditions between May 10 and May 15, 2026. Questions were categorized into retainers, mini-implants, and lateral cephalograms. Responses were independently evaluated by five board-certified orthodontists using a validated 5-point rubric (1 = Completely Inaccurate/Poor to 5 = Highly Accurate/Excellent). Inter-examiner reliability was assessed using a two-way random-effects, single-measures intraclass correlation coefficient (ICC (2,1)). Because data across the three AI platforms were paired per question, non-parametric paired comparisons were conducted using the Friedman test, followed by post-hoc pairwise Wilcoxon signed-rank tests with Bonferroni correction (a = 0.017). Effect sizes were calculated using Kendall’s W. Significant differences in overall performance were observed among the AI systems (χ² (2) = 48.32, p < 0.001, Kendall’s W = 0.396). ChatGPT achieved the highest overall mean score (4.31, 95% CI [4.20,4.42]), followed by Gemini (3.94, 95% CI[3.82,4.06]) and Copilot (3.67, 95% CI [3.53,3.81]). Post-hoc testing confirmed that ChatGPT significantly outperformed both Gemini ( p < 0.001) and Copilot ( p < 0.001). Cephalometric queries yielded the highest consistency across models, whereas mini-implant biomechanics exhibited the highest variability. Inter-examiner agreement was excellent (ICC (2,1) = 0.87,95%CI [0.81,0.92]). AI-based conversational models demonstrate promising potential as supplementary educational tools in orthodontics; however, variability in scientific reliability and clinical consistency persists. Careful professional supervision remains essential when utilizing AI-generated orthodontic information in educational or clinical settings. Limitations regarding dynamic model updates and evaluator subjectivity highlight that AI outputs should complement, rather than replace, professional clinical consultations.
Authors
- Bushra Anis (ORCID: https://orcid.org/0009-0006-1703-9624)
- Hiba Tabassum (ORCID: https://orcid.org/0009-0009-9219-9451)
- Laraib Nasir (ORCID: https://orcid.org/0009-0005-6532-8183)
- Anam Sattar (ORCID: https://orcid.org/0000-0003-4443-0275)
- Sadia Rizwan (ORCID: https://orcid.org/0009-0009-4290-8352)
- Imtiaz Ahmed
Institutions
- Dow University of Health Sciences (PK)
Publication Details
- Journal
- BMC Oral Health
- Published
- 2026-11-14
- DOI
- https://doi.org/10.1186/s12903-026-10439-7
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00