Exploring Exchangeability within Permutation Inference: Implications for Statistics and Data Science Educators

Permutation tests are a powerful and flexible approach to inference via resampling. As computational methods become more ubiquitous in the statistics curriculum, use of permutation tests has become more tractable. At the heart of the permutation approach is the exchangeability condition, which determines the appropriate null sampling distribution. We explore the exchangeability condition in the context of permutation tests, including settings where the exchangeability condition is not tenable. Our examples include two-sample comparisons of means, clustered designs, multiple linear regression, and two-sample comparisons of correlation. The different exchangeability challenges in each example allow further exploration of the definition and importance of the exchangeability requirement for permutation testing. We believe that understanding the exchangeability condition and how it relates to the null hypothesis gives students the skills to work effectively with permutation tests in more complicated modeling settings and to further their understanding of statistical inference. We close with pedagogical recommendations for instructors who wish to incorporate permutation inference into their curriculum as a means to increase student understanding of resampling-based inference.