Retrospective use of the Pragmatic-Explanatory Continuum Indicator Summary-2 trial design tool to assess design choices in randomized controlled trials: an empirical review

The Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) tool is designed to help researchers match their design decisions to the intended purpose of their trial. The intention of a trial can be "explanatory," which improves our understanding of how an intervention works, or "pragmatic," which supports decision-making in health care. Increasingly, the tool has been used for a secondary purpose: in systematic reviews. Here the tool is used to judge the level of "pragmatism" or "explanatoriness" of trials included in the review to aid the understanding of trial results. However, there is debate on the most reliable means of making this judgment. Sometimes judgements are made using one reviewer; other times, multiple reviewers. Our study evaluated interrater reliability of two methods of scoring trial publications using PRECIS-2: individual reviewer scores and pairs of reviewers agreeing on a consensus score. We also found that neither method we tested produced a reliable judgment using PRECIS-2, and the scores from two reviewers agreeing on a consensus were no more reliable than scores from a single reviewer. We performed an additional analysis that showed that simplifying the scoring from the original five-point scale to a three-point scale may give a more reliable judgment of the "pragmatism" or "explanatioriness" of published trials. This simpler method of scoring should be encouraged for retrospective use of PRECIS-2 in systematic reviews.