The effect of protocol preregistration on study replicability, reproducibility and recognition: a systematic review and meta-analysis

Background: The adoption of open science practices, particularly prospective protocol registration, is widely advocated to mitigate questionable research practices such as outcome switching and HARKing (Hypothesizing After the Results are Known). However, the empirical evidence regarding the impact of preregistration on studies’ quality and robustness remains fragmented. This systematic review and meta-analysis aimed to quantify the effect of protocol preregistration on study replicability, reproducibility, and academic recognition.Methods: We conducted a comprehensive literature search across multiple databases (including PubMed, EMBASE, PsycINFO, Scopus, and Web of Science) for studies published up to February 2025. We included meta-research studies reporting comparisons between (pre)registered and non-preregistered studies, in any field of research. Outcomes of interest included any assessment of reporting quality, availability of research materials, and data sharing for replicability; the proportion of statistically significant findings, effect sizes, and sample sizes for reproducibility; and number of citations and time to publication for publication recognition. Data were pooled using random effects meta-analysis, and outcomes that could not be quantitatively synthesized were analysed using Synthesis Without Meta-analysis (SWiM). Risk of bias was assessed with the RoBANS 2 tool. Results: The search yielded 2613 unique records, of which 49 cross-sectional retrospective meta-research studies met the inclusion criteria. No randomized controlled trials were identified. The included meta-research studies most commonly analysed randomized controlled trials in clinical medicine (n = 31), followed by studies in psychology-related fields (n = 7), systematic reviews (n = 5), and other research areas (n = 6). Meta-analysis revealed that preregistration is associated with an increase in transparent reporting (OR 2.51[1.76, 3.56]), a reduction in the reporting of favourable results (OR 0.49[0.29, 0.84]), and larger sample sizes. However, the effect on selective outcome reporting was inconclusive. The descriptive synthesis of outcomes not included in meta-analyses additionally supports the association between preregistration and improved transparent reporting, while also suggesting increased sharing of materials, code, and data.Conclusion: Preregistration is consistently associated with indicators of improved research replicability and reproducibility, including more transparent reporting, larger study sample sizes, and a lower likelihood of favourable findings. However, the certainty of the evidence remains very low because current evidence is derived almost exclusively from observational meta-research studies. More rigorous evaluations are needed to establish whether these associations reflect causal effects of preregistration.

Authors

Publication Details

Journal
MetArXiv (OSF Preprints)
Published
2026-11-17
DOI
https://doi.org/10.31222/osf.io/63qwr
Primary Topic
Meta-analysis and systematic reviews
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

The effect of protocol preregistration on study replicability, reproducibility and recognition: a systematic review and meta-analysis

Stephanie M. Zellers, Kimberley Elaine Wever, Cristina Priboi, Hanno Würbel
3 links
MetArXiv (OSF Preprints)
Meta-analysis and systematic reviews
preprint

The effect of protocol preregistration on study replicability, reproducibility and recognition: a systematic review and meta-analysis

Stephanie M. Zellers, Kimberley Elaine Wever, Cristina Priboi, Hanno Würbel
preprint en

Abstract

Background: The adoption of open science practices, particularly prospective protocol registration, is widely advocated to mitigate questionable research practices such as outcome switching and HARKing (Hypothesizing After the Results are Known). However, the empirical evidence regarding the impact of preregistration on studies’ quality and robustness remains fragmented. This systematic review and meta-analysis aimed to quantify the effect of protocol preregistration on study replicability, reproducibility, and academic recognition.Methods: We conducted a comprehensive literature search across multiple databases (including PubMed, EMBASE, PsycINFO, Scopus, and Web of Science) for studies published up to February 2025. We included meta-research studies reporting comparisons between (pre)registered and non-preregistered studies, in any field of research. Outcomes of interest included any assessment of reporting quality, availability of research materials, and data sharing for replicability; the proportion of statistically significant findings, effect sizes, and sample sizes for reproducibility; and number of citations and time to publication for publication recognition. Data were pooled using random effects meta-analysis, and outcomes that could not be quantitatively synthesized were analysed using Synthesis Without Meta-analysis (SWiM). Risk of bias was assessed with the RoBANS 2 tool. Results: The search yielded 2613 unique records, of which 49 cross-sectional retrospective meta-research studies met the inclusion criteria. No randomized controlled trials were identified. The included meta-research studies most commonly analysed randomized controlled trials in clinical medicine (n = 31), followed by studies in psychology-related fields (n = 7), systematic reviews (n = 5), and other research areas (n = 6). Meta-analysis revealed that preregistration is associated with an increase in transparent reporting (OR 2.51[1.76, 3.56]), a reduction in the reporting of favourable results (OR 0.49[0.29, 0.84]), and larger sample sizes. However, the effect on selective outcome reporting was inconclusive. The descriptive synthesis of outcomes not included in meta-analyses additionally supports the association between preregistration and improved transparent reporting, while also suggesting increased sharing of materials, code, and data.Conclusion: Preregistration is consistently associated with indicators of improved research replicability and reproducibility, including more transparent reporting, larger study sample sizes, and a lower likelihood of favourable findings. However, the certainty of the evidence remains very low because current evidence is derived almost exclusively from observational meta-research studies. More rigorous evaluations are needed to establish whether these associations reflect causal effects of preregistration.

MetArXiv (OSF Preprints)
Good health and well-being
Meta-analysis and systematic reviews
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon