Reproducibility debt: insights from practitioners

Abstract Context Reproducibility Debt (RpD) refers to the accumulative issues and challenges that hinder the ability to reproduce scientific outcomes, stemming from challenges inherent in scientific software code and data, including development, organisation, dissemination, and documentation. RpD may accumulate when researchers and scientific software practitioners engage in sub-optimal activities, often for short-term benefits, which ultimately compromise the reproducibility of research outcomes. Hence, it is essential to understand the factors that could lead scientific software teams to incur RpD and the strategies they adopt to prevent RpD in their projects. Objective This research aims to understand practitioners’ perspectives on RpD, the factors they perceive as contributing to it, their consequences, and the practices they adopt to address it. Methodology We present a mixed-methods study combining exploratory interviews with scientific software practitioners and a complementary survey conducted using our InsightRpD survey tool. Results In total, 23 practitioners participated in interviews and 59 completed the survey. We identified 69 practitioner-reported causes and 80 practitioner-reported effects associated with RpD, confirming and extending current knowledge. Of the 38 causes reported in our previous Systematic Literature Review (SLR), 31 were supported by evidence from this study. The empirical data also highlights 38 additional causes and 47 additional effects not captured in prior literature, which are associated with evolving software practices, such as automation, decentralised teams, and the use of generative AI. Additionally, we explore the mitigation strategies employed by practitioners to address and reduce the impact of RpD. Conclusion By identifying factors contributing to RpD, its associated consequences, and mitigation strategies from practitioner insights, this study provides evidence-based diagrams of practitioner-reported cause-effect relationships and practical approaches that advance the understanding and management of RpD, while informing future research and practice in Research Software Engineering (RSE) and software sustainability.