Retrospective comparison of three commercial artificial intelligence algorithms for detection of intracranial hemorrhage (ICH) in the emergency radiology department

Background Several commercial artificial intelligence (Al) algorithms are available for detecting intracranial hemorrhage (ICH), but independent clinical validation remains limited. Purpose To compare three commercially available Al algorithms for ICH detection on non- contrast head computed tomography (NCHCT). Material and Methods In this retrospective study, 4027 consecutive NCHCT examinations from a large emergency hospital in southwest Sweden were analyzed. Three Al algorithms were applied, with one vendor disclosing participation. Reports from two radiologists and all Al outputs were evaluated. Human-AI performance was assessed using an idealized logical OR model, assuming radiologists perfectly dismissed all false-positive Al flags to calculate system specificity. All positive or discrepant cases underwent expert manual review using two-tier consensus adjudication as the reference standard. Results Of 3902 evaluable examinations, 3517 were consistently negative by all readers. The remaining 385 cases underwent manual review, confirming ICH in 176 cases (4.5% prevalence) and excluding it in 209. Eight ICH cases missed by both radiologists were detected by at least one Al system. Aidoc performed best, with 90.3% sensitivity and 99.0% specificity. A simulated mathematical combination of Aidoc with a human reader increased sensitivity to 96.0% while maintaining 99.4% specificity ( P < 0.001), comparable to two radiologists. Conclusion Al performance varied substantially, with only one system demonstrating clinically relevant accuracy. Combining Al with human interpretation improved ICH detection and shows promise for future clinical implementation.