Evaluating Cardiovascular Devices Using Observational Analyses

It has long been accepted that observational analyses have an important role in evaluating use patterns and assessing the safety of different treatments, including cardiovascular devices, in clinical practice. With the proliferation of large electronic databases, there has been increasing interest in using observational analyses to also examine the comparative effectiveness of devices. However, these analyses are often met with skepticism because of concerns about whether they can generate credible evidence about causal effects. This is in part a result of the difficulty in meeting the assumptions necessary to interpret observational associations as causal effects and of the wide variability in analytic rigor. In this review, we outline frameworks and review methods for using observational analyses to answer questions about the effectiveness and safety of cardiovascular devices. We highlight the target trial framework as a practical tool for guiding observational comparative effectiveness analyses. We illustrate how the framework allows investigators planning and conducting observational analyses to organize their activities as responses to 3 prompting questions. First, what is the research question of the study (ie, "What do we want?")? Second, what are the resources-including background knowledge, research concepts, principles and methods, and available data-that can be brought to bear on the research question (ie, "What do we have?")? And third, what specific steps should be taken to use the available resources to answer the research question (ie, "What do we do?")? We focus our exposition on the evaluation of cardiovascular devices, for which randomized trial data are often limited and there is a strong need for real-world evidence. In this setting, real-world evidence is usually derived from observational comparisons of the treatment of interest with relevant comparator groups using data captured during routine care. A principled approach to the planning and conduct of observational analyses can improve the quality of real-world evidence generation and ensure that the results of observational studies on medical devices can support meaningful conclusions about the risks and benefits of new devices.