Heterogeneity in the Analysis of the ALSFRS-R in ALS Clinical Trials and its Effect on the Validity and Precision of Trial Conclusions

BACKGROUND AND OBJECTIVES: Disability rating scales play a pivotal role in clinical trials, but there is a notable lack of guidance on how to analyze these scales. Using amyotrophic lateral sclerosis as a case study, our aim was to explore how disability rating scales have been analyzed in completed clinical trials and to assess how these different approaches influence both the risk of false-positive findings and the statistical power to detect true treatment effects. METHODS: We searched PubMed and Embase to systematically identify randomized, placebo-controlled clinical trials using the revised ALS functional rating scale (ALSFRS-R) as primary end point, with ≥20 randomly assigned patients and ≥12-weeks of follow-up. Data were extracted on the statistical analysis approaches and strategies for handling missing data. Variability in statistical methods was mapped to the various research questions that the trials aimed to address. A simulation study assessed how each statistical method influenced validity (false-positive rate) and precision (statistical power), using the Ceftriaxone trial data set to model a realistic trial scenario. RESULTS: Our analysis included 45 randomized clinical trials, comprising a total sample size of 7,338 patients, and identified 39 distinct statistical methods using a mixture of longitudinal and cross-sectional techniques. Most trials (55.6%) did not use all available (longitudinal) ALSFRS-R measurements, resulting in suboptimal utilization of patient data and reduced statistical precision. Applying the different statistical methods to the same trial data set resulted in large differences in the estimated treatment effect size, ranging from a negative 1.33 to a positive 2.33 SD difference. Among the methods used, 38.9% (95% CI 24.8%-55.1%) were at risk of increasing false-positive rates, potentially contributing to the erroneous advancement of ineffective treatments. Statistical power of valid strategies varied widely, ranging from 17.9% to 78.2%. DISCUSSION: Our results demonstrate considerable variability in statistical methods, with the choice of method able to influence the estimated treatment effects, potentially resulting in misleading conclusions and uncertainty about treatment effects. This limits the interpretability and comparability of clinical trials and influences clinical decision-making and drug development. Establishing statistical consensus recommendations could improve the utility of disability scales in clinical trials and accelerate progress toward effective therapies for neurodegenerative diseases.