Strengthening the Rigor of Research Syntheses: Insights From Open Science to Build Cumulative Knowledge

Abstract: Industrial-organizational psychologists (IOPs) provide both theoretical and practical insights into work phenomena, aiming to benefit organizations, their members, and fellow researchers. To ensure these insights are robust and accurate, IOPs must rely on a robust, reliable knowledge base, such as those found in research syntheses (e. g., systematic reviews and meta-analyses). Yet, just as threats to validity can undermine the ability of primary data researchers to deliver rigorous and relevant findings, validity threats, such as selection bias, inconsistent coding, analytic flexibility, and incomplete reporting, can compromise the utility and robustness of research syntheses. Weaknesses in research syntheses are particularly problematic, as these syntheses hold a revered status as the “cornerstone of evidence-based management.” In this primer, we outline the major stages in a typical research synthesis life cycle (i. e., searching literature, coding research, analyzing data, reporting and archiving results), identify common pitfalls, and provide literature-driven open-science recommendations to circumvent these weaknesses.