Large Language Models in Clinical Neurology: A Scoping Review

Background: Large language models (LLMs) are increasingly explored for clinical applications in neurology, yet their real-world utility, safety, and optimal implementation remain uncertain. We systematically reviewed the literature to characterize current applications, evaluate evidence quality, and identify knowledge gaps regarding LLM use in clinical neurology. Methods: Following PRISMA guidelines, we searched PubMed, Embase, Scopus, Web of Science, and CENTRAL from January 1, 2022 through February 1 2026. for peer-reviewed studies evaluating LLM applications in clinical neurology. We included studies using large language models for clinically relevant neurology tasks from text or multimodal inputs. Two independent reviewers screened records, extracted data, and assessed risk of bias using the QUADS-AI. We synthesized evidence narratively across application domains, validation approaches, and model performance. Results: Thirty-six studies (published 2023-2026) spanning 8 neurology subspecialties met inclusion criteria; 13 were simulation or feasibility studies, 17 analyzed retrospective clinical data, and 6 reported prospective clinical validation. Proprietary models predominated; 7 studies used retrieval-augmented generation (RAG) and 3 used agentic frameworks. Performance was highest for constrained tasks, including binary diagnostic classification (area under the curve, AUC 0.75-0.94), information extraction (F1 score, 0.89-0.90), patient education question answering (accuracy, 68%-97%), and ischemic stroke thrombectomy decision support (AUC, 0.92). Open-ended case-based classification showed lower accuracy (42%-54%). Safety signals included hallucinations and fabricated citations, overconfident recommendations, and poor calibration; risk of bias was rated high in all included studies. Conclusion: LLMs show promise for selected neurology workflows, but current evidence is early, heterogeneous, and limited by high risk of bias and scarce prospective validation. Clinical translation will likely require RAG and agentic architectures that can plan multi-step tasks, retrieve guidelines and local protocols, verify and calibrate outputs, and produce structured, auditable recommendations with source attribution, with clinician oversight and prospective evaluation. Primary Funding Source: This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai and supported by the Clinical and Translational Science Awards (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences. Research reported in this publication was also supported by the Office of Research Infrastructure of the National Institutes of Health under award number S10OD026880 and S10OD030463. Registration: PROSPERO CRD420251082465.

Authors

Institutions

Publication Details

Journal
Neurology Clinical Practice
Published
2026-11-19
DOI
https://doi.org/10.1212/cpj.0000000000200687
Citations
3
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
2.91

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Large Language Models in Clinical Neurology: A Scoping Review

Mahmud Omar, Girish N. Nadkarni, Yiftach Barash, Alon Gorenshtein et al.
3 citations
Neurology Clinical Practice
Artificial Intelligence in Healthcare and Education
2.91
article

Large Language Models in Clinical Neurology: A Scoping Review

Mahmud Omar, Girish N. Nadkarni, Yiftach Barash, Alon Gorenshtein, Kamel Shihada, Eyal Klang
article en
3 citations

Abstract

Background: Large language models (LLMs) are increasingly explored for clinical applications in neurology, yet their real-world utility, safety, and optimal implementation remain uncertain. We systematically reviewed the literature to characterize current applications, evaluate evidence quality, and identify knowledge gaps regarding LLM use in clinical neurology. Methods: Following PRISMA guidelines, we searched PubMed, Embase, Scopus, Web of Science, and CENTRAL from January 1, 2022 through February 1 2026. for peer-reviewed studies evaluating LLM applications in clinical neurology. We included studies using large language models for clinically relevant neurology tasks from text or multimodal inputs. Two independent reviewers screened records, extracted data, and assessed risk of bias using the QUADS-AI. We synthesized evidence narratively across application domains, validation approaches, and model performance. Results: Thirty-six studies (published 2023-2026) spanning 8 neurology subspecialties met inclusion criteria; 13 were simulation or feasibility studies, 17 analyzed retrospective clinical data, and 6 reported prospective clinical validation. Proprietary models predominated; 7 studies used retrieval-augmented generation (RAG) and 3 used agentic frameworks. Performance was highest for constrained tasks, including binary diagnostic classification (area under the curve, AUC 0.75-0.94), information extraction (F1 score, 0.89-0.90), patient education question answering (accuracy, 68%-97%), and ischemic stroke thrombectomy decision support (AUC, 0.92). Open-ended case-based classification showed lower accuracy (42%-54%). Safety signals included hallucinations and fabricated citations, overconfident recommendations, and poor calibration; risk of bias was rated high in all included studies. Conclusion: LLMs show promise for selected neurology workflows, but current evidence is early, heterogeneous, and limited by high risk of bias and scarce prospective validation. Clinical translation will likely require RAG and agentic architectures that can plan multi-step tasks, retrieve guidelines and local protocols, verify and calibrate outputs, and produce structured, auditable recommendations with source attribution, with clinician oversight and prospective evaluation. Primary Funding Source: This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai and supported by the Clinical and Translational Science Awards (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences. Research reported in this publication was also supported by the Office of Research Infrastructure of the National Institutes of Health under award number S10OD026880 and S10OD030463. Registration: PROSPERO CRD420251082465.

Neurology Clinical PracticeVol. 17(1)
Beth Israel Deaconess Medical Center (US), Mount Sinai Health System (US), Harvard University (US), Bar-Ilan University (IL), Mount Sinai Medical Center (US), Icahn School of Medicine at Mount Sinai (US)
Georgia Clinical and Translational Science Alliance, National Institutes of Health, National Center for Advancing Translational Sciences
Good health and well-being
Openalex Percentile: Top 10%
Artificial Intelligence in Healthcare and Education
2.91
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon