Can the All of Us sample be reweighted to mirror a nationally representative sample? A comparison of mortality predictors

BACKGROUND: Participants in the All of Us research study differ from the U.S. population in myriad characteristics, limiting the generalizability of findings. A statistical reweighting tool to improve generalizability would enhance the scientific value of the data. METHODS: To account for differences between All of Us and the nationally representative 1999-2018 National Health and Nutrition Examination Survey (NHANES), we generated selection weights using four models incorporating sociodemographic, self-reported health, and clinical characteristics. We assessed covariate balance and compared predictors of all-cause mortality in weighted All of Us to NHANES using the ratio of hazard ratios (RHRs), where an RHR of one indicates unbiased estimates in (weighted) All of Us relative to NHANES. RESULTS: Weighting improved balance on measured variables between All of Us and NHANES. Among the four weighting models, the most complex model which included sociodemographic, health, and clinical variables and their interactions achieved HRs in All of Us most similar to those in NHANES. For example, the RHR for hypertension for unweighted All of Us vs. NHANES (RHR=1.5; 95% CI=1.4 to 1.7) was reduced to 1.2 (95% CI=0.9 to 1.5) after applying weights from the clinical-interaction model. Even in this model, 17 of 35 HRs evaluated diverged by >20% (RHR <0.8 or >1.2) between weighted All of Us and NHANES. CONCLUSIONS: Predictors of mortality in All of Us differ from those in the U.S. population both in their distribution and in their associations with mortality. Reweighting can mitigate selection bias, but no model we tested comprehensively achieved generalizability.