Granular by design: a fine-grained NER approach to context-aware medication extraction

Introduction While structured electronic health records (EHRs) provide a baseline for medication history, they often fail to capture clinically important contextual information about medication use that is documented only in free-text clinical notes. A significant portion of critical clinical information – such as unreported adverse events or over-the-counter supplement usage – resides exclusively in unstructured free-text clinical notes. Our objective was to develop and evaluate a fine-grained context-aware Named Entity Recognition (NER) framework capable of transforming medication-related information from unstructured Hungarian clinical narratives into structured data suitable for clinical research. Methods We developed a fine-grained context-aware Named Entity Recognition (NER) framework using the spaCy NLP library. Clinical documents were sourced from the UDBD-Health Research Data Repository of the University of Debrecen. A dedicated fine-grained annotation scheme was designed to categorize medication mentions into 4 contextual classes: Current Medication, Discontinued/No Medication, Adverse Events, and Allergies (CAVE). Documents were pre-annotated using rule-based vocabularies and manually refined in Label Studio. To evaluate model generalization and obtain a robust, leakage-free performance estimate, we employed 5-fold cross-validation with an inner validation split, in which model selection via early stopping used a validation subset drawn only from the training folds, while each outer held-out fold was reserved exclusively for final evaluation. To address the pronounced imbalance between medication context classes, we applied document-level oversampling of minority-class training examples. Model performance was assessed using Precision, Recall, and F 1 -score, reported as mean ± standard deviation across the five folds, and compared against a rule-based baseline built from the same vocabulary-matching procedure used for pre-annotation. Results Two types of models were trained for medication information extraction. The fine-grained context-aware model (CME) achieved an overall F 1 -score of 0.944 ± 0.001 across five-fold cross-validation. At the entity level, CURRENT_MED yielded the highest F 1 -score (0.958 ± 0.001), followed by CAVE_MED (0.900 ± 0.023), NO_MED (0.820 ± 0.005), and AE_MED (0.689 ± 0.026), the latter exhibiting the greatest fold-to-fold variance. Applying document-level oversampling of minority-class examples (CME OS) resulted in an overall F 1 -score of 0.946 ± 0.002, with modest improvements in the minority-class F 1 -scores. The coarse-grained model (ME), trained on the single-entity MEDICATION recognition task, attained an F 1 -score of 0.968 ± 0.001 with minimal variance, substantially outperforming a rule-based vocabulary-matching baseline on the same task (F 1 = 0.860 ± 0.003), which showed high Precision but comparatively low Recall. The models have been deployed into the institutional data pipeline, enabling automated structuring of medication documents from over 700,000 documents per year. Discussion This study presents a fine-grained context-aware medication extraction system for Hungarian clinical documents. The developed models facilitate the transformation of unstructured medication narratives into structured medication data, providing a foundation for downstream pharmacological, epidemiological, and real-world evidence studies. Limitations include institutional data specificity, underrepresentation of adverse-event contexts, and document- rather than patient-level cross-validation splitting, and the lack of independent parallel annotation, which prevented calculation of inter-annotator agreement. Future work will focus on corpus expansion, inter-institutional validation, and the inclusion of additional attributes such as dosage and duration.