Finite energy resolution, correlations between bins and non-nested hypotheses

Abstract We show that the finite energy resolution of a detector does not lead to a correlation between the bins of the observed spectrum unless a correlation is already present in the original spectrum. Our argument not only applies to the energy resolution but more generally to any case in which the probability distribution functions of the event and detector response are a Gaussian (or Poisson) and a multinomial, respectively. This is in contrast with a recent claim that such an effect not only occurs, but also significantly alters the distribution of $$\Delta \chi ^2,$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>Δ</mml:mi> <mml:msup> <mml:mi>χ</mml:mi> <mml:mn>2</mml:mn> </mml:msup> <mml:mo>,</mml:mo> </mml:mrow> </mml:math> the usual test statistic defined as the difference between two $$\chi ^2$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msup> <mml:mi>χ</mml:mi> <mml:mn>2</mml:mn> </mml:msup> </mml:math> computed under different hypotheses. We obtain a simple expression for the variance of $$\Delta \chi ^2$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>Δ</mml:mi> <mml:msup> <mml:mi>χ</mml:mi> <mml:mn>2</mml:mn> </mml:msup> </mml:mrow> </mml:math> in the case of non-nested hypotheses, which is valid even if correlations between bins are present. While derived with the mass ordering in mind, our results regarding the distribution of $$\Delta \chi ^2$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>Δ</mml:mi> <mml:msup> <mml:mi>χ</mml:mi> <mml:mn>2</mml:mn> </mml:msup> </mml:mrow> </mml:math> in the presence of (intrinsic) correlations also hold for any simple non-nested hypotheses.