Defining and Estimating Outcomes Directly Averted by a Vaccination Program when Rollout Occurs Over Time

During the COVID-19 pandemic, estimating the total deaths averted by vaccination was of great public health interest. Instead of estimating total deaths averted by vaccination among both vaccinated and unvaccinated individuals, some studies empirically estimated only "directly averted" deaths among vaccinated individuals, typically suggesting that vaccines prevented more deaths among unvaccinated and vaccinated individuals than directly among vaccinated individuals only, due to the indirect effect. Here, we define the causal estimand to quantify outcomes "directly averted" by vaccination-i.e., the impact of vaccination for vaccinated individuals, holding vaccination coverage fixed-for vaccination at multiple time points, which is a lower bound on the total outcomes averted when the indirect effect is non-negative. We develop an unbiased estimator for the causal estimand in a one-stage randomized controlled trial (RCT) and explore the bias of a popular "hazard difference" estimator frequently used in empirical studies. We show that even in an RCT, the hazard difference estimator is biased if vaccination has a non-null effect, as it fails to incorporate the greater depletion of susceptibles among the unvaccinated individuals. In simulations, the overestimation is small for averted deaths when infection-fatality rate is low, as for many important pathogens. However, the overestimation can be large for averted infections given a high basic reproduction number and a high vaccine efficacy against infection. Additionally, we define and compare estimand and estimators for avertible outcomes (i.e., outcomes that could have been averted by vaccination, but were not due to failure to vaccinate). Future studies can explore the identifiability of the causal estimand in observational settings.