A conditioning tactic that increases design sensitivity in observational block designs

Abstract In an observational block design, there are I blocks of J individuals, typically with one treated individual and J−1 controls; however, unlike a randomized block design, individuals were not randomly assigned to treatment or control. To be convincing, an observational block design must demonstrate that an ostensible treatment effect is not actually a consequence of small or moderate unmeasured biases of treatment assignment in the absence of a treatment effect. It is known that weighting to ignore blocks with a small range of responses increases the ability to distinguish a treatment effect from a bias in treatment assignment—that is, it increases the design sensitivity. Here, it is shown that a new tactic further increases design sensitivity. The new tactic involves a conditional statistic, such that blocks with moderately large ranges are considered conditionally given that the treated individual has either the largest or smallest response in the block. The new tactic is explored: (i) in terms of an asymptotic measure, the design sensitivity, (ii) in simulation of the power of a sensitivity analysis in finite samples, and (iii) in an example. Adaptive inference is briefly discussed. An R package weightedRank implements the method, contains the data, and reproduces the empirical results.