Using virtual experiments to improve data analysis

Abstract In the data analysis of measurements, the simplified assumption of homoscedastic Gaussian noise is often made to account for the random fluctuations between observations. This may be an inadequate assumption which can deteriorate the results of a data analysis. Repeated measurements, which can be used to infer the true distribution of the data, may be inaccessible, making the true distribution hard to find. In such circumstances, thoroughly designed virtual experiments (VEs) can mimic real and possibly complex measurement processes to infer the true data distribution, which can subsequently be accounted for in an improved analysis of real observations. We explore the potential benefit of such an approach in terms of a metrological application, the tilted-wave interferometer (TWI). Our VE for the TWI yields not just the mean of the data, but also their physically modelled, random fluctuations arising in repeated observations. We use the virtual data to derive a statistical data model that includes correlations and heteroscedasticity. In applying a Bayesian data analysis procedure utilising said statistical model in conjunction with a vague prior for the quantity of interest, virtual data with a known ground truth are analysed and the quality of the resulting estimates are assessed. In addition, a comparison is carried out to the often-employed, simplified approach assuming homoscedastic, independent noise. We observe a significant improvement in the results when a more adequate statistical model for the data is utilised, along with a reliable uncertainty quantification. The work proposes the idea to extend the utilisation of a VE to inferring the noise characteristics of real observations, in turn leading to significantly improved data analysis procedures. The potential benefit is demonstrated to be substantial in terms of the considered metrological case study. Future research is discussed, including other ways that VEs could be used to further improve data analysis.