A New Approach to the Nonparametric Behrens–Fisher Problem With Compatible Confidence Intervals

We propose a new method to address the nonparametric Behrens-Fisher problem, allowing for unequal distribution functions across the two samples. The procedure tests the null hypothesis <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:semantics> <mml:mrow><mml:msub><mml:mi>H</mml:mi> <mml:mn>0</mml:mn></mml:msub> <mml:mo>:</mml:mo> <mml:mi>θ</mml:mi> <mml:mo>=</mml:mo> <mml:mstyle><mml:mrow><mml:mn>1</mml:mn></mml:mrow> </mml:mstyle> <mml:mo>/</mml:mo> <mml:mstyle><mml:mrow><mml:mn>2</mml:mn></mml:mrow> </mml:mstyle> </mml:mrow> <mml:annotation>$\mathcal {H}_0: \theta = \nicefrac {1}{2}$</mml:annotation></mml:semantics> </mml:math> , where <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:semantics><mml:mrow><mml:mi>θ</mml:mi> <mml:mo>=</mml:mo> <mml:mi>P</mml:mi> <mml:mo>(</mml:mo> <mml:mi>X</mml:mi> <mml:mo><</mml:mo> <mml:mi>Y</mml:mi> <mml:mo>)</mml:mo> <mml:mo>+</mml:mo> <mml:mstyle><mml:mrow><mml:mn>1</mml:mn></mml:mrow> </mml:mstyle> <mml:mo>/</mml:mo> <mml:mstyle><mml:mrow><mml:mn>2</mml:mn></mml:mrow> </mml:mstyle> <mml:mi>P</mml:mi> <mml:mo>(</mml:mo> <mml:mi>X</mml:mi> <mml:mo>=</mml:mo> <mml:mi>Y</mml:mi> <mml:mo>)</mml:mo></mml:mrow> <mml:annotation>$\theta = \text{P}(X<Y) + \nicefrac {1}{2}\text{P}(X=Y)$</mml:annotation></mml:semantics> </mml:math> denotes the Mann-Whitney effect. Apart from the trivial case of one-point distributions, no restrictions are imposed on the underlying data distribution. The test is derived by evaluating the ratio of the true variance <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:semantics><mml:msubsup><mml:mi>σ</mml:mi> <mml:mi>N</mml:mi> <mml:mn>2</mml:mn></mml:msubsup> <mml:annotation>$\sigma _N^2$</mml:annotation></mml:semantics> </mml:math> of the Mann-Whitney effect estimator <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:semantics> <mml:msub><mml:mover><mml:mi>θ</mml:mi> <mml:mo>̂</mml:mo></mml:mover> <mml:mi>N</mml:mi></mml:msub> <mml:annotation>$\widehat{\theta }_N$</mml:annotation></mml:semantics> </mml:math> to its theoretical maximum, as derived from the Birnbaum-Klose inequality. Through simulations, we demonstrate that the proposed test effectively controls the type-I error rate under various conditions, including small and unbalanced sample sizes, and different data-generating mechanisms. Notably, it provides better control of the type-I error rate than the widely used Brunner-Munzel test, particularly at small significance levels such as <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:semantics><mml:mrow><mml:mi>α</mml:mi> <mml:mo>=</mml:mo> <mml:mn>0.005</mml:mn></mml:mrow> <mml:annotation>$\alpha = 0.005$</mml:annotation></mml:semantics> </mml:math> . We further construct range-preserving compatible confidence intervals and show that they exhibit improved coverage compared to the confidence intervals compatible to the Brunner-Munzel test. Finally, we illustrate the application of the method in a clinical trial example.