ChatGPT Performs Worse on USMLE-Style Ethics Questions Compared to Medical Knowledge Questions
Both versions of ChatGPT performed more poorly on medical ethics questions compared to medical knowledge questions. GPT-4 significantly outperformed GPT-3.5 on overall accuracy and exhibited a significantly lower response variability in answer choices. This underscores the need for ongoing assessment of ChatGPT versions for medical education.
Authors
- Jessica Hecht
- Clyde B. Schechter (ORCID: https://orcid.org/0000-0003-0010-2827)
- Sabrina Kentis (ORCID: https://orcid.org/0009-0003-9256-8057)
- Sunit P. Jariwala
- Tessa Danehy
Institutions
- Albert Einstein College of Medicine (US)
- Montefiore Medical Center (US)
Publication Details
- Journal
- Applied Clinical Informatics
- Published
- 2024-08-29
- DOI
- https://doi.org/10.1055/a-2405-0138
- Citations
- 9
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 1.08