Cumulative Differences Between Paired Samples

ABSTRACT Paired samples are observations from two populations, commonly with each observed response from one population corresponding to an observed response from the other population at the same value of an ordinal covariate. Such pairs of responses at the same covariate values are called “matched pairs.” Graphing cumulative differences between the two populations reveals differences in responses as a function of the covariate. The slope of the secant line connecting two points on the graph displays the average difference over the wide interval of covariate values between the points. (“Average” means “weighted average” if the samples are weighted.) A Kuiper‐style statistic summarizes into a single scalar the differences over all covariate values. Kuiper's metric is the absolute value of the total difference in responses between the two populations, totaled over the interval of covariate values for which the absolute value is greatest. Dividing the total difference by the total weight ensures this normalized total becomes the (weighted) average over all covariate values when taking the total over the entire range of the covariate. These cumulative statistics are notably nonparametric, unlike traditional methods such as reliability diagrams or parametric or semi‐parametric regressions, which typically obscure significant differences due to their parameter settings.