Individual Treatment Effect Modeling for AKI Alerts: A Randomized Trial

Background: Individual treatment effect (ITE) models predict the likelihood of benefit of an intervention for each individual in a population. They enable targeted automated alerts for conditions such as AKI in order to reduce alert fatigue while preserving alert benefit. Methods: A prior randomized trial by our group evaluating the use of informational AKI alerts showed a modest effect on process outcomes but no change in AKI progression, dialysis or death. Based on data from that trial, we used ITE modeling to determine phenotypic characteristics that predict benefit from an alert and expressed the multivariable prediction in the form of an “uplift score”. We then conducted a prospective randomized trial of ITE-targeted AKI alerts. Patients were randomized to “Recommend”: higher scores receive an alert and lower scores do not receive alerts vs. “Anti-recommend”: higher scores do not receive an alert and lower scores receive an alert (Fig. 1). The trial was stopped early due to a pre-specified futility analysis based on the primary outcome. Results: 2046 patients were randomized, mean age 65, 51% female, 19% black, and 12% Hispanic. The primary composite outcome occurred in 19% of patients in the Recommend group vs 18.5% in the Anti-Recommend group (Risk Ratio (RR)=1.03, 95% CI 0.86-1.23 p=0.78). However, randomization to Recommend led to a 31% risk decrease of readmission at 30 days (12.3% vs. 17.9%; RR=0.69, 95% CI 0.56–0.85, p=<0.001) and 18% decreased risk of renal consult (17.4% vs. 21.2%; RR=0.82, 95% CI 0.69–0.98, p=0.03)(Fig 2). Conclusion: This trial using ITE modeling showed no difference in the primary outcome between the intervention arms but did show that randomization to algorithm-recommended alerts decreased hospital readmissions and the rate of renal consult. Funding: NIDDK Support