Evaluating Artificial Intelligence (AI) Tools in Clinical Pulpal Scenarios

Background: This study aimed to compare the responses of various AI-based chatbots to different scenarios related to pulpal diseases and to analyze the reliability of these chatbots. Materials and Method: The declaration of Helsinki conducted the study and did not require ethical approval. A total of 20 open-ended clinical case questions involving various pulpal diseases were created and submitted to ChatGPT, Google Gemini, DeepSeek and Microsoft Copilot. The responses were scored on a scale from 1 (very poor) to 5 (excellent) using the Modified Global Quality Scale (MGQS), based on expert evaluation. Data were analyzed using IBM SPSS Statistics version 22.0 and both the Friedman test and Pearson correlation analysis were performed. Results: A strong and statistically significant correlation was found between ChatGPT and both DeepSeek (p < .001) and Microsoft (p < .001). A moderate but significant correlation was also observed between ChatGPT and Google (p = .019), as well as between Google and Microsoft (p < .018). In contrast, DeepSeek showed a weak and non-significant correlation with Google and Microsoft (p = .089). Descriptive statistics indicated that DeepSeek provided the highest and most consistent scores, whereas Google and Microsoft yielded more variable evaluations. However, according to the Friedman test results, there was no statistically significant difference in MGQS scores among the four AI systems (p = .461). Conclusion: ChatGPT produced more consistent responses than the other AI systems, while DeepSeek provided the highest and most stable responses. However, no statistically significant differences were observed among the systems based on MGQS scores.