The effect of protocol preregistration on study replicability, reproducibility and recognition: a systematic review and meta-analysis
Background: The adoption of open science practices, particularly prospective protocol registration, is widely advocated to mitigate questionable research practices such as outcome switching and HARKing (Hypothesizing After the Results are Known). However, the empirical evidence regarding the impact of preregistration on studies’ quality and robustness remains fragmented. This systematic review and meta-analysis aimed to quantify the effect of protocol preregistration on study replicability, reproducibility, and academic recognition.Methods: We conducted a comprehensive literature search across multiple databases (including PubMed, EMBASE, PsycINFO, Scopus, and Web of Science) for studies published up to February 2025. We included meta-research studies reporting comparisons between (pre)registered and non-preregistered studies, in any field of research. Outcomes of interest included any assessment of reporting quality, availability of research materials, and data sharing for replicability; the proportion of statistically significant findings, effect sizes, and sample sizes for reproducibility; and number of citations and time to publication for publication recognition. Data were pooled using random effects meta-analysis, and outcomes that could not be quantitatively synthesized were analysed using Synthesis Without Meta-analysis (SWiM). Risk of bias was assessed with the RoBANS 2 tool. Results: The search yielded 2613 unique records, of which 49 cross-sectional retrospective meta-research studies met the inclusion criteria. No randomized controlled trials were identified. The included meta-research studies most commonly analysed randomized controlled trials in clinical medicine (n = 31), followed by studies in psychology-related fields (n = 7), systematic reviews (n = 5), and other research areas (n = 6). Meta-analysis revealed that preregistration is associated with an increase in transparent reporting (OR 2.51[1.76, 3.56]), a reduction in the reporting of favourable results (OR 0.49[0.29, 0.84]), and larger sample sizes. However, the effect on selective outcome reporting was inconclusive. The descriptive synthesis of outcomes not included in meta-analyses additionally supports the association between preregistration and improved transparent reporting, while also suggesting increased sharing of materials, code, and data.Conclusion: Preregistration is consistently associated with indicators of improved research replicability and reproducibility, including more transparent reporting, larger study sample sizes, and a lower likelihood of favourable findings. However, the certainty of the evidence remains very low because current evidence is derived almost exclusively from observational meta-research studies. More rigorous evaluations are needed to establish whether these associations reflect causal effects of preregistration.
Authors
- Stephanie M. Zellers (ORCID: https://orcid.org/0000-0001-8927-3483)
- Kimberley Elaine Wever (ORCID: https://orcid.org/0000-0003-3635-3660)
- Cristina Priboi (ORCID: https://orcid.org/0000-0001-8498-0509)
- Hanno Würbel (ORCID: https://orcid.org/0000-0002-2934-3010)
Publication Details
- Journal
- MetArXiv (OSF Preprints)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.31222/osf.io/63qwr
- Primary Topic
- Meta-analysis and systematic reviews
- Type
- preprint