Evaluating LLMs In Dental Clinical Decision Support

Clinical decision support systems leveraging artificial intelligence have the potential to offer invaluable assistance to doctors, enhancing treatment outcomes. This study aims to assess the capacity of LLMs, ChatGPT 4.0, ERNIE 4.0 and DeepSeek-V3 for oral clinical diagnosis and treatment. Twenty standardized oral clinical cases were independently processed by ChatGPT 4.0, ERNIE 4.0, and DeepSeek-V3. Model outputs were evaluated against senior residents' responses using four metrics: accuracy, readability, clinical interpretability, and safety. In all scenarios, mean accuracy rates were 85.2% (ChatGPT 4.0), 86.4% (ERNIE 4.0), 87.3% (DeepSeek-V3), and 92.6% (senior residents). Total scores demonstrated significantly higher performance for senior residents compared to all LLMs (p < 0.05). Residents also outperformed models in safety, clinical interpretability, and readability across all cases. While LLMs showed acceptable performance in oral clinical decision-making, their accuracy and clinical utility lagged behind senior residents. Limitations likely stem from LLMs' reliance on general-purpose training data and limited real-time medical updates. Future advancements could prioritize specialized medical training and dynamic data integration to bridge this gap.