Large language models fail to reliably predict emergent catheterization laboratory activation from prehospital electrocardiograms

Background Rapid and accurate electrocardiogram (ECG) interpretation is essential for timely identification of ST-elevation myocardial infarction (STEMI) and activation of reperfusion pathways in emergency care. Objectives To evaluate the diagnostic performance of multimodal LLMs in identifying prehospital ECGs warranting emergent catheterization laboratory activation. Methods We performed a retrospective analysis of 615 ECGs from 270 emergency medical service patient encounters (EMS) with concern for acute myocardial infarction. The reference standard was cardiology activation of the STEMI pathway for emergent angiography. LLM-based image interpretation (three models) and ECG machine algorithm interpretations were compared. Sensitivity, specificity, positive predictive value, negative predictive value, and overall accuracy were calculated. Results Gemini demonstrated the highest sensitivity (95.3%; 95% CI 91.7–97.3) but extremely poor specificity (9.4%), indicating a high false-positive rate. ChatGPT and Claude showed moderate sensitivity (68.1% and 67.2%) with limited specificity (42.3% and 46.5%). The ECG machine algorithm demonstrated more balanced performance, with sensitivity of 67.7% (95% CI 61.4–73.4) and higher specificity (64.2%) than all LLMs. Conclusions Multimodal LLM interpretation of prehospital ECGs demonstrated clinically unreliable performance for identifying ECGs warranting emergent cardiac catheterization laboratory activation when benchmarked against real-world cardiology activation decisions. Although some models achieved high sensitivity, poor specificity resulted in excessive false-positive activation recommendations. These findings suggest that general-purpose LLMs are not appropriate for ECG-based catheterization laboratory activation decisions in time-sensitive cardiopulmonary care workflows.