AI assisted detection of large vessel occlusion on non-contrast CT: multinational validation and reader study

OBJECTIVE: Early detection of large vessel occlusion (LVO) on non-contrast CT (NCCT) could accelerate stroke triage, but NCCT based artificial intelligence (AI) algorithms lack multinational validation and evidence of impact on clinician performance. We aimed to validate a machine learning based LVO detection algorithm across multinational cohorts and evaluate its impact on clinicians' diagnostic performance. METHODS: In this retrospective study, an AI algorithm was validated using independent cohorts from Korea (n=723; consecutive; 127 with LVO) and the US (n=240; case-control; 120 with LVO). Standalone performance was assessed using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. A multi-reader, multi-case crossover study involving eight physicians evaluated diagnostic performance with and without AI assistance. Clinical utility was quantified using net reclassification improvement (NRI), number needed to screen (NNS), and benefit-to-harm ratio (BHR). RESULTS: Standalone AI achieved AUC values of 0.963 (95% CI 0.946 to 0.975) in the Korean cohort and 0.899 (95% CI 0.858 to 0.939) in the US cohort. AI assistance significantly increased pooled AUC values from 0.718 to 0.852, sensitivity from 46.6% to 63.7%, and specificity from 91.9% to 94.9% (all P<0.001). AI gave an NRI of 5.5%, an NNS of 18.2, and a BHR of 2.89. Automation bias analysis showed a reliance level BHR of 9.25. CONCLUSION: This multinational validation confirmed the robustness and generalizability of a machine learning based LVO detection algorithm on NCCT. AI assistance significantly improved clinicians' sensitivity and diagnostic accuracy, suggesting its potential as a supportive tool in acute stroke triage.