Comparison of disease risk score methods to study treatment effect heterogeneity: a simulation study

Estimating treatment effects across disease risk scores (DRS) is a common approach for assessing treatment effect heterogeneity in randomized trials. When external models are unavailable, the optimal approach for internally fitting the DRS model remains uncertain. This simulation study compares three internal derivation approaches, evaluating bias of estimated treatment effects within DRS-defined strata. We simulated trials under varying treatment effects (odds ratios [OR] of 1, 0.8, 0.5) and treatment-covariate interactions. We fit DRS models on 1) controls-only, 2) the full sample ignoring treatment assignment, and 3) a random split-sample of 50% of the controls, who were removed from the second stage of analysis. Additional simulations varied outcome incidence, sample size, randomization ratio, and the true DRS c-statistic. Bias decreased with the split-sample method (overall percent bias (OPB) 7.7% for OR = 0.8 with interactions) compared to the controls-only (OPB = 15.6%) and full-sample methods (OPB = 22.1%). Bias decreased more with split-sample than controls-only and full-sample methods with larger sample sizes, higher outcome incidence, greater treated-to-control ratios, and larger c-statistics. These findings suggest split-sample methods may be the preferred approach to estimate treatment effect heterogeneity by the DRS in trials with sufficient data to support stable prediction modeling.