The Challenges Of Replication Research In Sports And Exercise Science

Communication with original authors is important for replication as clarifications can identify any confounding factors that could affect the replication outcome. PURPOSE: To reflect on the challenges of conducting a large replication project. METHODS: As part of a large replication project, we contacted original authors to request deidentified raw data or full summary statistics. Each replication study was preregistered and made available for review by original authors. RESULTS: Of the 156 original authors contacted, 13% shared data which is in line other replication projects. Ethical concerns, lack of data access, and destroyed data were reasons not to share while some authors simply did not want to assist. Of the 34 preregistrations sent, 18% were approved without comments, 12% had comments and 70% were not reviewed at all. Of 15 studies needing methodological clarifications, 73% responded. Of the 133 studies in the pool for replication, 62% of studies reported effect size estimates while the availability of other statistical information is shown in table 1 below. Table 1. - Availability of key statistical test information in 133 selected original studies. Means of each group Standard deviations of each group Exact p-value Test statistic(t or F) Reported 101 (76%) 101 (76%) 91 (68%) 59 (44%) Not reported 321 (28%) 32 (24%) 42 (32%) 74 (56%) *Total number of studies = 133. Percentages are reported in the brackets.1 6 studies reported mean differences only but not means of each group The availability of statistical information is directly related to the ability to closely replicate a study. Poor communication from original authors can make the attempted replication of a theoretical construct from an original study problematic. Poor reporting of statistical information can also lead to overuse of conservative estimates given the lack of means, variances and effect size estimates required for power analysis. CONCLUSION: Communication from original authors is essential for high quality replications but our results show a great degree of difficulty in replicating sport and exercise science studies based on peer-reviewed manuscripts. Better reporting practices and increased engagement by original authors will reduce the challenge of assessing the original study being replicated. Irish Research Council Government of Ireland Postgraduate Scholarship Programme