Investigating impacts of health policies using staggered difference-in-differences: the effects of adoption of an online consultation system on prescribing patterns of antibiotics

Abstract Since the COVID-19 pandemic, most general practices in England have adopted online consultation (OC) systems for easier access and demand management. However, there are concerns that easier access could increase demand relating to low-risk minor illness, with consequential changes in prescribing patterns. Using GP practice-level data we investigate effects of adoption of an OC system in English general practice on antibiotic prescribing rates. The target estimand is the average effect of adoption on the outcome in each month post-adoption within groups of practices defined by month of adoption. Estimation is using a doubly robust staggered difference-in-differences approach. We also estimate an effect aggregated across all adopting practices and across time since adoption. Overall, we find evidence that adoption of an OC system results in higher antibiotic prescribing rates, with our primary analysis indicating an overall 5.1% increase. However, using recently proposed methods, we find that our estimates are sensitive to possible violations of the key identification assumption, conditional parallel trends. When allowing for relatively small violations of this assumption, we would no longer conclude that there is statistically significant evidence of an effect and smaller in magnitude effects become more plausible, which may be considered reassuring for health policy.