Performance of ChatGPT in dentistry
Artificial Intelligence-based language model, ChatGPT, has gained significant traction due to its communicative interface and the relevance of the responses generated. This tool exhibits tremendous potential to be utilized in dentistry for dental education, and possibly as a clinical decision support system. Hence, it is imperative to evaluate the accuracy of the model in relation to the responses generated for dental-related queries and outline the limitations for its use in clinical practice. This study aims to evaluate the performance of ChatGPT-generated responses to questions from multiple dental specialties for their accuracy, completeness, and relevance. Methods: This multi-centric study involved 27 subject experts from nine dental specialties of various institutions and 2 heads of institutions. A total of 243 questions were formulated and the answers generated by ChatGPT (version: 3.5) were rated in terms of accuracy (6-point Likert), completeness (4-point Likert), and relevance (5-point Likert). Results: The mean accuracy of the ChatGPT-generated answers was 4.61 (SD 1.575), with a median of 5.33. For completeness, the mean score was 2.01 (SD 0.793), and the median was 2.33. Regarding relevance, a mean of 3.13 (SD 1.590) and a median of 3.67 were obtained. The highest ratings were observed for answers related to Oral Medicine and Radiology, as well as for open-ended questions, and questions labelled as easy in terms of difficulty. Conclusion: The promising results observed in the study promote the application of ChatGPT for retrieving dental information. However, it is crucial to exercise caution and seek advice from a qualified healthcare for dental health-related queries. Further large-scale testing of the model is necessary before incorporating it into dental clinical practice.
Authors
- S. Gokul (ORCID: https://orcid.org/0000-0002-6757-6691)
- Sharmila Tapashetti (ORCID: https://orcid.org/0000-0002-7009-1388)
- Bindiya Narang (ORCID: https://orcid.org/0000-0002-6233-4132)
- Gokul Sridharan (ORCID: https://orcid.org/0000-0002-6119-7068)
- Amit Vasant Mahuli (ORCID: https://orcid.org/0000-0002-4746-5935)
- Sangita Singh (ORCID: https://orcid.org/0000-0003-1559-6372)
- Anita Parushetti
- Sugandha Verma (ORCID: https://orcid.org/0000-0002-3638-5868)
- Arpita Rai (ORCID: https://orcid.org/0000-0001-9078-0642)
- Anka Sharma (ORCID: https://orcid.org/0000-0003-3998-5655)
- Gaurav Shah
- Tushar Tushar
- Jeyaseelan Augustine (ORCID: https://orcid.org/0000-0003-2074-8991)
- Nishant Mehta (ORCID: https://orcid.org/0000-0001-6273-4892)
- Arvind Sivakumar (ORCID: https://orcid.org/0000-0002-5504-2262)
- Vijay Yadav (ORCID: https://orcid.org/0000-0003-3372-5625)
- Panchali Batra
- Nishat Sultan (ORCID: https://orcid.org/0000-0001-7167-1934)
- Farheen Sultan
- Deborah Sybil (ORCID: https://orcid.org/0000-0002-8793-3051)
- Simpy Amit Mahuli (ORCID: https://orcid.org/0009-0003-0798-2000)
- Sridhar Kannan (ORCID: https://orcid.org/0000-0003-2082-4595)
- Ranjit J. Injety (ORCID: https://orcid.org/0000-0003-1403-8336)
- Ashish Jain (ORCID: https://orcid.org/0000-0003-1010-0294)
- Priyanshu Kumar Shrivastava (ORCID: https://orcid.org/0000-0003-0078-3757)
- Sanjay Singh (ORCID: https://orcid.org/0009-0001-4731-6569)
- Madhu Ranjan (ORCID: https://orcid.org/0000-0002-6601-6959)
- Vivek Gupta (ORCID: https://orcid.org/0000-0003-0714-6136)
- Shabina Sachdeva (ORCID: https://orcid.org/0000-0002-9637-0887)
- Neelam Singh (ORCID: https://orcid.org/0000-0003-3128-7764)
- Sayani Roy (ORCID: https://orcid.org/0000-0003-0556-4780)
Institutions
- Christian Medical College, Vellore (IN)
- Rajendra Institute of Medical Sciences (IN)
- Sri Dharmasthala Manjunatheshwara College of Dental Sciences & Hospital (IN)
- Maulana Azad Institute of Dental Sciences (IN)
- Christian Medical College (IN)
- G Pulla Reddy Dental College & Hospital (IN)
- Patanjali Research Foundation (IN)
- Bhagwan Mahaveer Jain Hospital (IN)
- Sinhgad Dental College and Hospital (IN)
- Govt. Dental College & Hospital (IN)
- Jamia Millia Islamia (IN)
- All India Institute of Medical Sciences (IN)
- Kamineni Institute of Dental Sciences (IN)
Publication Details
- Journal
- Brazilian journal of oral sciences/Brazilian Journal of Oral Sciences
- Published
- 2025-05-03
- DOI
- https://doi.org/10.20396/bjos.v24i00.8674954
- Citations
- 1
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.65