Inherent safety risks of guidelines derived from “cause-agnostic” randomized controlled trials: a review

Physicians rely on guidelines derived from randomized controlled trials (RCTs) to determine whether interventions are safe and effective for specific diseases or causal mechanisms. In critical care, however, well-powered RCTs demonstrating clear evidence of benefit and subsequently incorporated into clinical guidelines have repeatedly been abandoned after causing harm. This strong evidence of benefit from robust RCTs or meta-analyses followed by unexpected harm during clinical application, generates an “evidence deployment paradox”, which has been attributed to unavoidable biological heterogeneity rather than to structural flaws in trial design. Recent analysis has shifted attention to the architecture of the RCT itself, indicating that the paradox arises from a mathematical set-mixing problem introduced by a modification of the original RCT model when the field adopted a trial design in which enrollment was decoupled from causal diagnosis or the targeted causal mechanism and replaced by consensus-derived, disease- and cause-agnostic triage thresholds. The triage selects broadly shifting mixtures of diseases with potentially different treatment effects. We term this design the cause-agnostic randomized controlled trial. Although cause-agnostic RCT can generate internally valid estimates within trial participants, their results may paradoxically be unsafe to incorporate into clinical guidelines, even when applied in the same hospitals and using the same entry criteria as the source trials. Structural causal modeling provides tools to mitigate this evidence-deployment paradox. Such modeling makes explicit the causal assumptions underlying enrollment and can identify when, and to what extent, a trial has deviated from cause-and-effect logic of the original randomized trial design.