QUADAS-3: Updated Tool to Evaluate Risk of Bias and Applicability Concerns in Diagnostic Test Accuracy Studies

Introduction: The QUADAS‐2 tool, published in 2011, was designed to evaluate the risk of bias and applicability of diagnostic test accuracy (DTA) studies. The publication reporting QUADAS-2 has been cited over 12, 000 times and it is the recommended tool to assess risk of bias and applicability of studies for major HTA organizations. Although feedback on QUADAS-2 has generally been positive, some signaling questions have been identified as problematic and the tool could be improved based on features included in more recently developed tools. Objectives: To update QUADAS-2 to develop the new QUADAS-3 tool. Methods: We established a core-group of methodological experts to lead the development of the QUADAS-3 tool supported by a wider steering group. We followed the following steps: Summarised modifications made to QUADAS-2 for the Cochrane Handbook Web-based survey of reviewers that have used QUADAS-2 Considered developments from more recent tools in terms of tool structure and implementation Undertook a review of methodological studies that had evaluated QUADAS-2 Undertook a review of 50 Cochrane DTA reviews to highlight challenges with the assessment of applicability We have produced a draft tool which has undergone piloting. The results of the piloting, which also included a comparison of the use of signalling questions with signalling statements, was used to inform the final version of the tool. Results The new tool follows a similar structure to the QUADAS-2 tool but with some major updates. Key changes include: An option to define separate synthesis questions rather than just a single review question A new section on defining the ideal test accuracy trial for each synthesis question Assessment of risk of bias and applicability at the accuracy estimate level rather than the study level A change in answers to signaling questions to include options of “probably yes” and “probably no” and to replace “unclear” with “no information” Replacement of “Flow and Timing domain” with new “Analysis” domain Changes to some signaling questions Inclusion of a section for judging overall risk of bias and applicability (across domains) Conclusions: The QUADAS-3 tool incorporates several changes compared to the previous version (QUADAS-2) which we hope will improve its validity, usability, and usefulness. QUADAS-3 will be introduced at the conference and the results of piloting discussed.