An Analysis of Variance of the Pantheon+ Dataset: Systematics in the Covariance Matrix?

We investigate the statistics of the available Pantheon+ dataset. Noticing that the χ2 value for the best-fit ΛCDM model to the real data is small, we quantify how significant its smallness is by calculating the distribution of χ2 values for the best-fit ΛCDM model fit to mock Pantheon+-like datasets, using the provided covariance matrix. We further investigate the distribution of the residuals of the Pantheon+ dataset with respect to the best-fit ΛCDM model, and notice that they scatter less than would be expected from the covariance matrix but find no significant kurtosis. These results point to the conclusion that the Pantheon+ covariance matrix is over-estimated. One simple interpretation of these results is a ∼7% overestimation of errors on SN distance moduli in Pantheon+ data. When the covariance matrix is reduced by subtracting an intrinsic scatter term from the diagonal terms of the covariance matrix, the best-fit χ2 for the ΛCDM model achieves a normal value of 1580 and no deviation from ΛCDM is detected. We further quantify how consistent the ΛCDM model is with respect to the modified data with the subtracted covariance matrix using model-independent reconstruction techniques such as the iterative smoothing method. We find that the standard model is consistent with the data. There are a number of potential explanations for this smallness of the χ2, such as a Malmquist bias at high redshift, or accounting for systematic uncertainties by adding them to the covariance matrix, thus approximating systematic uncertainties as statistical ones.