Evaluation of multiple generative large language models on neurology board-style questions
LLMs-particularly ChatGPT-5 and ChatGPT-4o-exceeded resident performance on text-based neurology board-style questions across subspecialties and cognitive levels. Gemini 2.5 showed substantial gains over v1 but with domain-uneven scaling. Given weak confidence calibration, LLMs should be integrated as supervised educational adjuncts with ongoing validation, version governance, and transparent metadata to support safe use in neurology education.
Authors
- Alok Dabi (ORCID: https://orcid.org/0000-0002-6231-7287)
- Xiang Fang (ORCID: https://orcid.org/0000-0001-9716-8253)
- Peter McCaffrey (ORCID: https://orcid.org/0000-0002-4450-8143)
- Jorge Mario Rodríguez-Fernández (ORCID: https://orcid.org/0000-0003-4888-2041)
- Dan Hier
- Chih-Ying Li
- Mohammad Almomani
- James Weatherhead
- Vijaya Valaparla
Institutions
- Missouri University of Science and Technology (US)
- University of Illinois Chicago (US)
- The University of Texas Medical Branch at Galveston (US)
Publication Details
- Journal
- Frontiers in Digital Health
- Published
- 2026-01-05
- DOI
- https://doi.org/10.3389/fdgth.2025.1737882
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00