Bayesian adaptive trial designs for evaluating low-risk programmatic changes for quality improvement in health services: a simulation study

Quality improvement in health service programmes often involves introducing small, low-risk programmatic changes, such as modifying workflows, to incrementally improve outcomes, which accumulate over time to have significant overall gains in quality and efficiency. Although these changes are common in health services, they are rarely evaluated using statistically rigorous designs, partly because conventional randomised trials are perceived as inefficient for detecting modest effects. This study was motivated by a vision screening and referral programme and focuses on evaluating the modest but important impacts of low-risk QI interventions. To advance data-driven approaches for achieving quality improvement, we used a simulation study to explore the use of Bayesian adaptive trial designs to compare two variants of programmatic changes that yield small improvements in outcomes. The study examined key adaptive design features, including interim analysis frequency, prior specification, and early stopping rules for efficacy and equivalence. Changes in error rates, sample size, and bias were assessed across scenarios with small effect sizes ranging from 0% to 5%. The findings were used to configure an ideal trial design that prioritises rapid identification of the more effective programmatic change while minimizing the risk of adopting an inferior one. The recommended trial design incorporates a sceptical prior, a stringent stopping rule for efficacy, and a relaxed criteria to stop for equivalence. Under this design, a marginal improvement as small as 1% could be detected with high probability using considerably fewer participants than would be required under conventional, fixed-size randomised controlled trials. Bayesian adaptive trial designs offer a feasible approach for evaluating low-risk, incremental QI interventions in high-throughput service settings. Their use may support more efficient, data-driven decision-making when modest improvements are expected and the consequences of incorrect adoption are limited.