We need reproducibility in our researches to increase the trust in what we do

When you look at a recipe and you want to try it, you would like to find out if others have tried it and produced similar results as that of the original chef. When you try it yourself and you come up with something good, then you know that the recipe is reproducible; therefore, you trust the recipe. The same goes for research. What is reproducibility in research? The short answer is the repeatability of research findings. When research can be duplicated using the same materials and methods by the original study in an attempt to yield the same results, that is a reproducible study. It is method-oriented. It plays a big part in advancing scientific research. The accurate reporting of the methodology can be the basis for rigorous testing of the findings and ultimately increased confidence and trust in the results. The US National Academy of Science, Engineering, and Medicine in 2020 emphasized reproducibility in its report, stating that “repeated findings of comparable results tend to confirm the veracity of an original scientific conclusion while repeated failures tend to throw the original conclusions to doubt.”[1] Furthermore, since research findings become the basis of health policies, it can have an impact on the lives of people. Many studies claim a significant result, but their findings cannot be reproduced. This has led to the journal Nature to conduct a survey “Is there a reproducibility crisis?” in 2016.[2] Of the 1,576 researchers, 52% said that there is a significant crisis. More than 70% of researchers have tried and failed to reproduce another scientist’s experiments. More than half failed to reproduce their own experiments. In obstetrics and gynecology, at least two researches have shown that research in the field has not shown reproducibility consistently. In the first study by Rauh et al., 208 of 300 studies had empirical data that can be assessed for reproducibility and none had a link to their protocols.[3] Only 10.58% provided a statement regarding data availability. The second study looked at the randomized controlled trials published in 2017–2019 in six top journals, including the American Journal of Obstetrics and Gynecology (AJOG), British Journal of Obstetrics and Gynecology (BJOG), Journal of American Medical Association (JAMA), Lancet, New England Journal of Medicine (NEJM), and Obstetrics and Gynecology.[4] Eighty percent of the 170 trials were adequately compliant with the Consolidated Standards of Reporting Trials manuscript guidelines. Ninety-eight percent reported pre-trial registration. Concordance between pre-trial registration and publication in terms of primary outcome was at 77%, secondary outcomes at 32%, and sample size at 60%. These point to much better data compared to the first study but still point to needing improvement. How can we as researchers improve our reproducibility? First, we need to identify what contributes to poor reproducibility. There have been several reports indicating the major reasons contributing to non-reproducibility.[2,5,6]Table 1 summarizes those points as well as some recommendations on how to address them.Table 1: Problems leading to lack of reproducibility and proposed solutionsMaking data openly available is probably the most controversial aspect of reproducibility. Many researchers hesitate to share data for the fear that they will be used incorrectly or unethically by others. We need to be knowledgeable of tools that are available to researchers that will enable them to store data and get credit when their data are used. What about journals? Journals have big roles to play in ensuring the trustworthiness and integrity in what they publish. The International Science Council (ISC) on its Occasional Paper published in 2021 recommends two items.[7] Co-publication of data is one answer to this. While this is laudable, there can be issues met with regard to sensitive data. As an interim step, journals should have clear statements relating to data access. For clinical trials, pre-registration is a requirement. The second lies in the peer review process. The simplest question each peer reviewer should be able to answer is “Is there enough detail given of the methods involved, and if necessary, is the data available, that is if I wanted to, I can reproduce this work?”.[7] The use of reporting guidelines and checklists that help researchers meet certain criteria when publishing studies is a big help according to the Academy of Medical Sciences in its symposium report.[6] In 2016, Cell, a leading biology journal, and its publisher Cell Press have adopted a new methods section so that the authors can be guided in clearly communicating how their studies or experiments have been conducted.[8] It goes by the acronym STAR which stands for Structured, Transparent, Accessible Reporting. It contains a Key Resources Table which highlights the reagents, organisms, strains, cell lines, and their sources and identifiers. Software, instrumentation, and source data essential to reproduce results are also included. In 2019, they came up with STAR Protocols which is an open-access protocol journal featuring step-by-step method details, how the experiment will be conducted, troubleshooting, expected outcomes, and the Key Resources Table. Needless to say, reproducibility is a responsibility of the researcher. It is the researcher’s responsibility to explain the uncertainty in the results and in the conclusions, to make proper use of statistical methods, and to describe their methods in the clearest, most accurate, and complete way. The journals and their publishers take part in promoting reproducibility also by having clear policies on data sharing and access, reporting of clear methodologies, use of reporting guidelines, and inclusion of these considerations in the peer review process. This discourse is a reminder to all of us – researchers in the field that reproducibility should be a way of life to be built in all of us. Once you learn it, it becomes a habit.