Evaluating artificial intelligence in the diagnosis of hip fractures: an analysis of sensitivity, specificity, positive and negative predictive values, and accuracy

Abstract Objectives To evaluate the diagnostic performance of Milvue SmartUrgences AI software for detecting hip fractures on radiographs obtained in the emergency department (ED). Methods This retrospective diagnostic accuracy study included consecutive patients aged ≥ 60 years undergoing hip and pelvis radiography at two EDs between June 2023 and September 2024. The index test was Milvue SmartUrgences, which classified radiographs as “YES,” “NO,” or “DOUBT.” The reference standard was established by a consensus panel of three musculoskeletal radiologists and two senior orthopaedic trauma surgeons, blinded to AI output. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1-score were calculated. The primary analysis included patients with definitive “YES” or “NO” classifications; “DOUBT” classifications were evaluated separately. Results Among 539 included patients, 487 received definitive AI classifications and were included in the primary analysis. The mean age was 83.4 years (SD 8.4), and 66% were female. The reference standard identified 191 hip fractures. Sensitivity was 97.4% (95% CI 94.0–98.9), and specificity was 86.5% (95% CI 82.1–89.9). PPV was 82.3% (95% CI 76.8–86.7), NPV 98.1% (95% CI 95.6–99.2), and accuracy 90.8% (95% CI 87.9–93.0). The F1-score was 89.2%. Conclusions Milvue SmartUrgences demonstrated high sensitivity and negative predictive value for hip fracture detection in a consecutive emergency department cohort. The software may provide useful decision support during acute radiographic assessment, but AI output should complement rather than replace clinical and radiological assessment.