Minimal clinically important differences and the illusion of precision

OBJECTIVE: Minimal clinically important differences (MCIDs) are used in palliative care trials to guide sample size calculations and to judge whether changes in patient-reported outcomes represent meaningful benefit. Several methods are used to estimate MCIDs. For the Total Symptom Distress Score (TSDS), two common approaches are the within-patient mean change and receiver-operating characteristic (ROC)-based method.MCIDs derived using these approaches often differ across otherwise similar studies. We examined whether this variability reflects limited sample size or whether commonly used ROC-based methods contribute to persistent instability. METHODS: Data for 186 participants were drawn from two published randomised controlled trials. MCIDs were estimated using (1) the within-patient mean change (anchored to minimal improvement) and (2) ROC analysis to maximise Youden's J. Stratified bootstrap resampling was performed at nominal sample sizes of 50, 100 and 150. RESULTS: With increasing sample size, variability in ROC-derived MCIDs decreased only modestly and remained substantial even in large samples. Multiple thresholds showed near-equivalent discriminatory performance. At n=150, multiple near-optimal thresholds were observed in up to 85.4% of bootstrap samples. The differences in the threshold values were clinically relevant. In contrast, mean-change MCIDs converged towards more consistent values (approximately -6.5 TSDS points), with markedly lower clinically-relevant variability. CONCLUSION: When MCIDs are used as fixed thresholds for trial design or interpretation, mean-change approaches are more reproducible across plausible samples, whereas ROC-based MCIDs show greater variability that can meaningfully affect trial conclusions. Caution is therefore warranted when using single ROC-derived MCIDs in palliative care trials.