Decoding Complications: Large Language Model-Based Classification of Free-Text Entries in Clinical Stroke Registries
Clinical registries capture adverse events (AEs) in free-text fields that are rarely analyzed. We developed a reproducible large language model (LLM)-assisted workflow using 11,216 free-text entries from 18,069 patients in the German Stroke Registry; 9,146 informative entries were retained after filtering. Embedding-based clustering showed poor separation, whereas an LLM-guided taxonomy identified 89 leaf labels. On the model-assisted 1,000-entry development set used for model and prompt selection, GPT-5.4 reached F1 92.4% (95% confidence interval [CI] 91.4-93.4). On 300 held-out entries, weighted F1 against clinician consensus was 96.5% (94.6-98.1) for frozen GPT-5.4 labels and 90.5% (87.5-93.5) for locally run Qwen 3.5 27B. For AE concepts with a comparable structured field, free text documented 1,345 additional instances (+19.6% over 6,867 structured instances); it also documented 7,546 instances of AE concepts without a structured equivalent, most often pneumonia, urinary tract infection and delirium. Beyond baseline covariates and the structured AE count, integrated complication concepts increased the area under the curve for prediction of good functional outcome by 0.025 (95% CI 0.021-0.028) and for death by 0.018 (0.014-0.021); gains were minimal after including the discharge National Institutes of Health Stroke Scale (NIHSS) score. On 1,979 public coronary-trial fragments from the Aggregate Analysis of ClinicalTrials.gov (AACT), the software ran end to end, demonstrating its technical applicability to a second clinical text corpus. In conclusion, this workflow enables structured analysis of clinical registry free text and complements structured adverse-event documentation.
Authors
- Taraneh Ebrahimi (ORCID: https://orcid.org/0000-0002-5471-4266)
- Christian Thielscher (ORCID: https://orcid.org/0000-0002-7109-4820)
- Gabor C. Petzold (ORCID: https://orcid.org/0000-0002-0145-8641)
- Felix Jürgen Bode (ORCID: https://orcid.org/0000-0003-4498-186X)
- Lara Klehr (ORCID: https://orcid.org/0009-0008-3429-8198)
- Louisa Nitsch (ORCID: https://orcid.org/0000-0002-5139-6848)
- Sebastian Stösser (ORCID: https://orcid.org/0000-0002-1182-705X)
- Hannah Asperger (ORCID: https://orcid.org/0000-0003-3253-7016)
- Omid Shirvani Samani (ORCID: https://orcid.org/0009-0006-6367-8742)
- Julius Nicolai Meißner (ORCID: https://orcid.org/0000-0002-8980-9337)
- Julia Layer (ORCID: https://orcid.org/0000-0001-5904-5316)
- Niklas Michael von Danwitz
- Johannes Weller
- Sach Mukherjee
- Franziska Dorn
- on behalf of the GSR-ET Investigators
Institutions
- University Hospital Bonn (DE)
- German Center for Neurodegenerative Diseases (DE)
Publication Details
- Journal
- Journal of Medical Systems
- Published
- 2026-11-16
- DOI
- https://doi.org/10.1007/s10916-026-02486-y
- Primary Topic
- Machine Learning in Healthcare
- Type
- article
- Field-Weighted Citation Impact
- 0.00