Multi-Agent Generative Artificial Intelligence for Transfusion Reaction Feature Extraction

BACKGROUND: Many transfusion reactions are not reported to hemovigilance systems, and the features needed to recognize them are often documented only in free-text clinical notes. We sought to develop and evaluate an open-source, multi-agent artificial intelligence (AI) pipeline to extract transfusion reaction features defined by the National Healthcare Safety Network (NHSN) Hemovigilance Module from clinical notes. METHODS: We curated 181 features from the NHSN protocol and developed a multi-stage pipeline that extracted features, their timing, and clinical details from free-text notes. Three open-weight large language models (Qwen 3.6-27B, GPT-OSS-120B, and Kimi K3) were applied to 50 notes from expert-written scenarios, published case reports, and MIMIC-IV discharge summaries. Two physicians independently evaluated every model output in a blinded interface. The primary analysis was restricted to feature occurrences with concordant physician assessments. RESULTS: Physicians agreed on 1,201 of 1,343 co-rated feature occurrences (89.4%; Krippendorff's alpha 0.73). Precision was above 0.98 for all models. Sensitivity was 0.774 (95% CI 0.745-0.800) for Kimi K3, 0.636 (0.604-0.667) for GPT-OSS-120B, and 0.624 (0.593-0.655) for Qwen 3.6-27B, and specificity ranged from 0.952 to 0.991. Omissions accounted for more than 94% of false negatives. Timing and clinical details were judged supported or reasonable in more than 92% and 97% of assessments, respectively. Results were consistent across note sources and in analyses including discordant assessments. CONCLUSION: Multi-agent AI pipelines provide a feasible method for extracting transfusion reaction signals from clinical notes with high precision. Our work may have applications in active hemovigilance, reaction classification, and big data research.