Evaluation of machine learning models for stroke risk prediction in atrial fibrillation: a systematic review and meta-analysis

Atrial fibrillation (AF) increases the risk of cerebrovascular accidents (CVA), underscoring the need for accurate risk prediction. Traditional tools like CHA₂DS₂-VASc use limited clinical variables, often yielding imprecise estimates. Machine learning (ML) can offer potential for improved prediction by integrating broader clinical data. This study assesses the methods, performance, and real-world applicability of ML models for CVA prediction in AF. A systematic search of PubMed, Embase, and Web of Science was conducted in September 2024. A random-effects meta-analysis on eligible studies pooled the predictive performance (AUROC) of ML models and the CHA₂DS₂-VASc score. Comparative analyses were conducted between the two, alongside subgroup analyses by ML algorithm type and validation strategy. This study was prospectively registered in PROSPERO (CRD42024539628). Seventeen studies were included. Tree-based models were most frequently used, followed by logistic regression and neural networks. The pooled AUROC for ML models was 0.66 (95% CI: 0.64–0.68), comparable to CHA₂DS₂-VASc (0.64; 95% CI: 0.62–0.66; P = 0.327). Sensitivity analyses confirmed robustness of estimates, and no significant differences were found between algorithm types. Models evaluated only on training data had significantly higher AUROCs than those using validation sets (P = 0.002), suggesting performance inflation. Moreover, only two studies employed external validation, highlighting a critical gap in methodological rigor. Notably, heterogeneity arising from diversity in model designs, populations, outcomes, and CVA prevalence (as identified by meta-regression) limited the generalizability of our findings. Current ML models perform comparably to traditional tools for CVA risk prediction in AF patients; however, clinical applicability warrants methodological standardization and external validation in diverse populations.