Evaluating the Performance and Fragility of Large Language Models on the Self-Assessment for Neurological Surgeons
While current LLMs demonstrate an impressive ability to answer neurosurgery board-like examination questions, their performance is markedly vulnerable to extraneous, distracting information. These findings underscore the critical need for developing novel mitigation strategies aimed at bolstering LLM resilience against in-text distractions, particularly for safe and effective clinical deployment.
Authors
- Anton Alyakin
- Douglas Kondziolka (ORCID: https://orcid.org/0000-0001-8925-7344)
- Karl L. Sangwon (ORCID: https://orcid.org/0000-0002-6941-3001)
- Eric K. Oermann (ORCID: https://orcid.org/0000-0002-1876-5963)
- Jin Ha Lee (ORCID: https://orcid.org/0000-0002-9007-514X)
- Daniel Alexander Alber
- Krithik Vishwanath (ORCID: https://orcid.org/0009-0000-6894-1796)
- Mrigayu Ghosh
Institutions
- Washington University in St. Louis (US)
- NYU Langone Health (US)
- Médecins Sans Frontières (US)
- Neurological Surgery (US)
- New York University (US)
- The University of Texas at Austin (US)
Publication Details
- Journal
- Neurosurgery
- Published
- 2025-12-08
- DOI
- https://doi.org/10.1227/neu.0000000000003878
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00