Abstract WP314: Natural Language Processing and AutoML for Predicting Post-Stroke Medication Adherence from Clinical Discharge Notes

Background: Post-stroke medication adherence is crucial for preventing recurrence and promoting optimal recovery. Early identification of patients at risk is critical for delivering precise interventions. Discharge summaries from electronic health records (EHRs) capture rich clinical narratives that may provide predictive signals for medication adherence. Objective: This study aimed to develop an automated prediction framework for post-stroke medication compliance by combining natural language processing (NLP) methods with automated machine learning (AutoML) using Medical Information Mart for Intensive Care (MIMIC-IV) discharge summaries. Methods: We identified ischemic stroke patients in the MIMIC-IV database using both ICD-9 and ICD-10 codes. Adherence status was first reviewed by one investigator and then verified by a nurse, who examined discharge summaries and follow-up notes for documentation of missed refills or nonadherence to prescribed medications. A total of 4,018 stroke patients were included, with 108 labeled as nonadherent and 3,910 as adherent in the MIMIC data set. Discharge summaries were preprocessed with NLP methods to generate text embeddings and modeled using AutoGluon’s AutoML framework. Performance was evaluated using stratified cross-validation, with metrics including AUROC, F1-score, and recall. Results: Compared across different models, the random forest achieved the best performance (AUROC, F1, and recall), outperforming the weighted ensemble and neural network models (Figure 1). Patients predicted to be nonadherent were often characterized by limited social support, cognitive impairment, and discharge to rehabilitation facilities, reflecting transition-of-care risks. Conclusion: Integrating NLP feature extraction with AutoML provides a scalable and interpretable framework for predicting post-stroke medication adherence from discharge summaries. This approach demonstrates the potential of leveraging unstructured clinical text and automated modeling pipelines to inform targeted interventions and improve secondary stroke prevention.