Assessing the feasibility and impact of clinical trial trustworthiness checks via an application to Cochrane Reviews: Stage 2 of the INSPECT-SR project

Systematic reviews collate evidence from randomized controlled trials (RCTs) to find out whether health interventions are safe and effective. However, it is now recognized that the findings of some RCTs are not genuine, and some of these studies appear to have been fabricated. Various checks for these "problematic" RCTs have been proposed, but it is necessary to evaluate these checks to find out which are useful and which are feasible. We applied a comprehensive list of "trustworthiness checks" to 95 RCTs in 50 systematic reviews to learn more about them and to see how often performing the checks would lead us to classify RCTs as being potentially inauthentic. We found that applying the checks led to concerns about the authenticity of around 1 in three RCTs. However, we found that many of the checks were difficult to perform and could have been misinterpreted. This might have led us to be overly skeptical in some cases. The findings from this study will be used, alongside other evidence, to decide which of these checks should be performed routinely to try to identify problematic RCTs, to stop them from being mistaken for genuine studies and potentially being used to inform health care decisions.

Authors

Institutions

Publication Details

Journal
Journal of Clinical Epidemiology
Published
2025-05-09
DOI
https://doi.org/10.1016/j.jclinepi.2025.111824
Citations
12
Primary Topic
Meta-analysis and systematic reviews
Type
article
Field-Weighted Citation Impact
16.78

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Assessing the feasibility and impact of clinical trial trustworthiness checks via an application to Cochrane Reviews: Stage 2 of the INSPECT-SR project

George Athanasios Antoniou, Andreas Lundh, David Ruben Teindl Laursen, Lisa Parker et al.
12 citations
Journal of Clinical Epidemiology
Meta-analysis and systematic reviews
16.78
article

Assessing the feasibility and impact of clinical trial trustworthiness checks via an application to Cochrane Reviews: Stage 2 of the INSPECT-SR project

George Athanasios Antoniou, Andreas Lundh, David Ruben Teindl Laursen, Lisa Parker, Ella Flemyng, Kenneth Färnqvist, David Nunan, Rickard Carlsson, Ashma Krishan, Alison Avenell, Kyle Alexander Sheldrick, Madelon van Wely, Calvin Heal, Kylie E Hunter, David N. Borg, Silvy Laporte, Steph Grohmann, Sarah F. Lensen, Sarah Cotterill, Eleftheria Patetsini, Zewen Lu, Nicholas J. L. Brown, Lyle C. Gurrin, Lisa Anne Bero, Sofia Tsokani, Lesley‐Anne Carter, Jack Dale Wilkinson, Antonia Marsden, Jill Alison Hayden, Martin Ringsten, Emma Sydenham, Rachel Richardson, Anna Lene Seidler, Clara Locher, Stephanie Weibel, Michael C Ferraro, Toby J Lasserson, Jamie J Kirkham, Katie Stocking, Constant Vinatier, Timothy Hugh Barker, Tianjing Li, Zachary Munn, Colby J. Vorland, Alessandra Alteri, Bassel H. Al Wattar, Gideon Meyerowitz‐Katz, Aidan G Cashin, Florencia Weber, Florian Naudet, Ewelina Rogozińska, Sarah Rhodes, Hugh Thomas, Jose Andrés Calvache, Robert Buhmann, Natasha Olsson, Barbara K. Redman, Neil Edward O'Connell, Chang Xu, Ben W. Mol, Rui Wang, Natalie Hyltse, Jianping Liu, Wentao Li, lim hee jung, Love Ahnström
article en
12 citations

Abstract

Systematic reviews collate evidence from randomized controlled trials (RCTs) to find out whether health interventions are safe and effective. However, it is now recognized that the findings of some RCTs are not genuine, and some of these studies appear to have been fabricated. Various checks for these "problematic" RCTs have been proposed, but it is necessary to evaluate these checks to find out which are useful and which are feasible. We applied a comprehensive list of "trustworthiness checks" to 95 RCTs in 50 systematic reviews to learn more about them and to see how often performing the checks would lead us to classify RCTs as being potentially inauthentic. We found that applying the checks led to concerns about the authenticity of around 1 in three RCTs. However, we found that many of the checks were difficult to perform and could have been misinterpreted. This might have led us to be overly skeptical in some cases. The findings from this study will be used, alongside other evidence, to decide which of these checks should be performed routinely to try to identify problematic RCTs, to stop them from being mistaken for genuine studies and potentially being used to inform health care decisions.

Journal of Clinical EpidemiologyVol. 184
The University of Sydney (AU), Dalhousie University (CA), Royal Women's Hospital (AU), University of Cauca (CO), Vita-Salute San Raffaele University (IT), Inserm (FR), The University of Melbourne (AU), University of the Sunshine Coast (AU), Beijing University of Chinese Medicine (CN), Second Military Medical University (CN), University of Southern Denmark (DK), Lund University (SE), University of Aberdeen (GB), University of Wollongong (AU), Aristotle University of Thessaloniki (GR), Linnaeus University (SE), Manchester Academic Health Science Centre (GB), Australian Sports Commission (AU), Odense University Hospital (DK), Karolinska Institutet (SE), University of Manchester (GB), UNSW Sydney (AU), University of Oxford (GB), MRC Clinical Trials Unit at UCL (GB), Manchester University NHS Foundation Trust (GB), Institut de Recherche en Santé, Environnement et Travail (FR), Indiana University Bloomington (US), Universitätsklinikum Würzburg (DE), National Centre for Clinical Research on Emerging Drugs (AU), Eastern Hepatobiliary Surgery Hospital (CN), Skåne University Hospital (SE), Amsterdam University Medical Centers (NL), Istituti di Ricovero e Cura a Carattere Scientifico (IT), Centre Hospitalier Universitaire de Saint-Étienne (FR), National Health and Medical Research Council (AU), Istituto di Ricovero e Cura a Carattere Scientifico San Raffaele (IT), University College London (GB), University of Rostock (DE), Anglia Ruskin University (GB), University of Colorado Anschutz Medical Campus (US), University of York (GB), Université de Rennes (FR), The University of Adelaide (AU), New York University (US), Brunel University of London (GB), Cochrane (GB), Neuroscience Research Australia (AU), University of Amsterdam (NL)
National Institute for Health and Care Research, Department of Health and Social Care
Climate action
Openalex Percentile: Top 1%
Meta-analysis and systematic reviews
16.78
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.