An Exploratory Analysis of ChatGPT Compared to Human Performance With the Anesthesiology Oral Board Examination: Initial Insights and Implications
ChatGPT generated SOE answers with comparable module ratings to anesthesiology fellows, as graded by 8 ABA oral board examiners. However, the ChatGPT answers were deemed subjectively inferior due to the length of responses and lack of focus. Future curation and training of an AI database, like ChatGPT, could produce answers more in line with ideal ABA SOE answers. This could lead to higher performance and an anesthesiology-specific trained AI useful for training and examination preparation.
Authors
- Tiffany Pham (ORCID: https://orcid.org/0000-0002-0308-4540)
- Lesley A. Hall (ORCID: https://orcid.org/0000-0002-9979-268X)
- Michael Gonzalez (ORCID: https://orcid.org/0000-0002-4957-2910)
- Alexander S. Doyal (ORCID: https://orcid.org/0000-0003-1238-4251)
- Ameer Ali (ORCID: https://orcid.org/0000-0001-9438-4903)
- Daniel J. Rosenkrans
- Daniel Winecoff
- Abhijit V. Lele (ORCID: https://orcid.org/0000-0002-6956-538X)
- Anthony N. Passannante
- David A. Zvara
- Samuel N. Blacker (ORCID: https://orcid.org/0000-0002-8775-1813)
- Bryan J. Hierlmeier (ORCID: https://orcid.org/0000-0001-6198-155X)
- Rachel M. Kacmar (ORCID: https://orcid.org/0000-0002-5874-9917)
- Courtney Hood
- Fei Chen (ORCID: https://orcid.org/0000-0003-2152-368X)
- Benjamin Cobb
- Anthony Plunkett
- Benjamin L. Antonio
- Robert S. Isaak
- Harendra Arora
- Kenneth Bradbury Brown
Institutions
- University of North Carolina at Chapel Hill (US)
- Womack Army Medical Center (US)
- University of Washington (US)
- Harborview Medical Center (US)
- Jackson Memorial Hospital (US)
- University of Mississippi Medical Center (US)
- University of Colorado Anschutz Medical Campus (US)
Publication Details
- Journal
- Anesthesia & Analgesia
- Published
- 2024-09-13
- DOI
- https://doi.org/10.1213/ane.0000000000006875
- Citations
- 13
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 1.56