Information quality assessment of DeepSeek artificial intelligence on orthodontic aligners: A reliability and validity study

Objectives: The objective of the study is to evaluate the information quality, reliability, and clinical utility of DeepSeek AI’s responses to frequently asked questions (FAQs) about orthodontic aligners. Material and Methods: Seventeen orthodontic aligner FAQs were systematically compiled from clinical expertise and public search analysis. Each question was presented twice to DeepSeek AI (version 3.1) with a 1-week interval. A blinded orthodontist evaluated all 34 responses using a modified 5-point Global Quality Scale (GQS). Intra-rater reliability was assessed through weighted Cohen’s kappa and intraclass correlation coefficient (ICC). AI performance was analyzed for proficiency (low/high-threshold criteria), test-retest reliability (ICC), and internal consistency (Cronbach’s alpha). Results: Expert intra-rater reliability was excellent (κ w = 0.89, ICC = 0.92). DeepSeek AI demonstrated exceptional performance: 100% of questions met low-threshold proficiency (GQS ≥4 both rounds), while 94.1% achieved high-threshold proficiency (GQS = 5 both rounds). Test-retest reliability was excellent (ICC = 0.98), and internal consistency was high (α = 0.96). Qualitative assessment revealed perfect scores in accuracy, clarity, and actionability, with minor reductions in completeness (4.97/5) and objectivity (4.94/5) for comparative questions. Conclusion: DeepSeek AI generates high-quality, reliable information for patient education about orthodontic aligners, demonstrating near-perfect proficiency and consistency. While it excels in factual content, minor limitations exist in balanced comparative analysis. These findings support the cautious integration of AI as a supplementary educational tool in orthodontic practice, with professional oversight recommended for clinical discussions.