Abstract 4141151: Coronary Artery Disease Polygenic Risk Scores with Equivalent Performance Produce Discordant Estimates of Individual Risk

Background: Dozens of polygenic risk scores (PRSs) have been developed to estimate coronary artery disease (CAD) risk. Some are available clinically. PRS performances are traditionally evaluated and compared at the population level. Whether they provide similar individual-level assessments of disease liability is a critical consideration for clinical implementation that remains uncharacterized. Research Question: How does individual-level risk prediction compare across CAD PRSs? Methods: Published CAD PRSs were curated from the PGS Catalog, and two novel PRSs were created from the results of a large GWAS meta-analysis. For each score, normalized individual risk scores were calculated for All of Us Research Program (AOU) participants. PRSs with equal population-level performance were identified by using Bayesian analysis of variance to compare calibration and discrimination (Brier score and AUROC) of generalized linear models of prevalent CAD. Among equivalently performing scores, the individual-level agreement between risk percentile estimates was tested with intraclass correlation (ICC) and Light’s Kappa. This approach was replicated in the Penn Medicine Biobank (PMBB) and UCLA ATLAS Biobank. Results: 50 PRSs were calculated for 171,095 AOU participants. When included in a model of prevalent CAD, 48 scores had practically equivalent Brier scores and AUROCs (region of practical equivalence = 0.02). Across these scores, 84% of AOU participants had at least one score in both the top and bottom risk quintile. Continuous agreement of individual risk predictions from the 48 scores was poor, with an ICC of 0.351 (95% CI; 0.349, 0.352). Agreement between two statistically equivalent scores was moderate, with an ICC of 0.646 (95% CI 0.643, 0.649). Light’s Kappa, used to evaluate consistency of assignment to high-risk percentile cutoffs, did not exceed 0.56 (interpreted as ‘fair’) across statistically and practically equivalent scores. Repeating the analysis among 41,193 PMBB and 50,748 UCLA participants yielded different sets of statistically and practically equivalent scores which also lacked strong individual agreement. Conclusions: When tested across three diverse biobanks, CAD PRSs that performed equivalently at the population level produced highly variable individual estimates of risk. Approaches to clinical implementation of CAD PRSs need to consider the potential for discordant individual risk estimates from otherwise indistinguishable scores.