Performance of artificial intelligence for fracture detection: an umbrella review and second-order-meta-analysis

Abstract Fractures affect approximately 178 million people annually, and artificial intelligence (AI) tools for fracture detection are increasingly approved by regulators. We conducted an umbrella review and second-order meta-analysis (PROSPERO CRD42024516414), searching six databases to 4/06/2025 for systematic reviews or meta-analyses of AI fracture detection using any imaging modality. Screening, extraction, and quality assessment were performed in duplicate. We assessed review quality using AMSTAR-2, primary-study overlap using corrected covered area, and certainty using GRADE. Fifty reviews were included (25 systematic reviews and 25 meta-analyses; 387 primary studies). Twenty-one meta-analyses (269 primary studies) contributed to second-order meta-analysis, with slight overall overlap (4.31%). Pooled AI sensitivity was 0.897 (95% CI 0.882–0.911; 95% prediction interval 0.753–0.962) and specificity 0.904 (95% CI 0.891–0.916; prediction interval 0.824 to 0.950). Comparative evidence showed no consistent differences for AI, clinicians, and AI-assisted clinicians: however, this data was sparse and inconclusive. No review reported adverse events or other safety outcomes. Certainty was very low/low for almost all outcomes; all meta-analyses were rated critically low/low on AMSTAR-2. Although published estimates suggest high diagnostic accuracy, real-world and comparative performance remain uncertain. Prospective external validation, multi-reader-multi-case studies, and randomised trials are needed before broad clinical adoption.