Artificial intelligence chatbots versus dentists: a comparative knowledge assessment on traumatic dental injury management

The aim of this study is to conduct a comparative analysis of the guideline-based knowledge levels of dentists and artificial intelligence(AI)-powered chatbots (ChatGPT-4o and Gemini) regarding the emergency management of traumatic dental injuries (TDIs). A 20-item multiple-choice questionnaire, developed based on the trauma guidelines recommended by the American Association of Endodontists (AAE), was administered to both AI-powered chatbots (ChatGPT-4o and Gemini) and practicing dentists. The guideline-based knowledge level and consistency of the AI responses were evaluated based on the collected data. Furthermore, the knowledge levels of the AI systems were statistically compared to those of the dentists, using a significance level of p < 0.05 and a 95% confidence interval. Upon analysis of the questionnaire responses, ChatGPT-4o provided significantly more correct answers than both dentists and Gemini in 17 out of the 20 questions (p < 0.05). There was a statistically significant difference in guideline-based knowledge levels among the groups (p = 0.001; p < 0.05). The rate of high-level knowledge demonstrated by ChatGPT-4o (100%) was statistically significantly greater than that of both dentists (12.6%) and Gemini (3.4%) (p < 0.05). ChatGPT-4o exhibited similar internal consistency score to Gemini in terms of reliability. ChatGPT-4o and Gemini may be considered potential sources of information in the context of TDIs. Although ChatGPT-4o provided significantly more accurate and consistent responses compared to Gemini, it is not entirely sufficient. Further research involving AI models specifically developed for the field of endodontics is necessary to address current limitations.