Open Science, Multidisciplinarity, and the Culture of Error: towards the Methodological Review Board

This presentation, delivered by Giulia Calignano at the ITalian Rreproducibility Network (ITRN) Annual Meeting 2025, explores the current transformations in psychological research brought about by the Open Science movement. It highlights the importance of multidisciplinarity, the need for a new understanding of scientific error, and proposes the creation of a Methodological Review Board (MRB) as a practical step toward improving research quality. Open Science emphasizes transparency, accessibility, and collaboration. Its success relies on integrating insights from various disciplines, such as ethics, philosophy, psychology, and statistics. Ethical considerations are crucial, especially when it comes to data sharing, informed consent, and the protection of participants. Philosophical perspectives enrich the conversation by offering critical tools and historical awareness, while statistical and psychometric methods remain essential for robust study design. One of the central themes of this presentation is the way science handles errors. Errors are unavoidable in research, yet they are often stigmatized, which discourages openness and learning. Many researchers fear professional consequences, such as reputational damage or public criticism, when mistakes occur. A more productive approach involves recognizing different kinds of errors, from honest mistakes to questionable research practices (QRPs) and more serious issues like fabrication or plagiarism. Building a positive culture around error involves understanding its causes, learning from it, and encouraging transparency rather than punishment. One of the most promising proposals is the creation of a Methodological Review Board. Unlike ethics committees, the MRB would support researchers in refining their study designs before data collection begins. This early feedback would improve methodological rigor, reduce errors, and offer guidance - particularly valuable for early-career researchers. It would also help foster a community of experts committed to methodological excellence. However, the idea is not without challenges. There are concerns that such boards could introduce bureaucracy, suppress innovation, or delay research. To avoid these pitfalls, it is crucial that MRBs are advisory rather than controlling, and that they remain responsive to the needs of the scientific community. At the University of Padova’s Department of Developmental and Social Psychology (DPSS), a pilot MRB is currently being developed by Psicostat, an interdisciplinary group of psychologists and statistitians (https://psicostat.dpss.psy.unipd.it/) - the first of its kind in Italy for the field of psychology. This initiative, inspired by a similar model at Eindhoven University proposed by Daniel Lakens, is supported by the Psicostat group and aligns with the department’s commitment to Open and Citizen Science. The goal is to promote a culture of transparency and collaboration by encouraging practices like preregistration and methodological consultation. The implementation plan includes public outreach events, an internal survey to assess researchers’ attitudes, and a pilot phase involving volunteer research teams. The process is grounded in international collaboration and adheres to national guidelines on research integrity. In conclusion, establishing a Methodological Review Board represents a meaningful step toward higher-quality, more ethical, and transparent research. For this model to succeed, it must be implemented gradually, with active input from the academic community, and be based on a culture of trust, support, and shared responsibility.