Comparison of Approaches for Variance Estimation with Application to a Survey on Discrimination

Usually, Statistical Institutes and Research Centers present the results of surveys in a “standardized” way, that is, the estimator of the parameter of interest and the associated measure of accuracy. This implies the calculation of the variance of the estimator, which is typically done in two possible ways: analytically or with replication methods. The analytic way requires the availability of the information of auxiliary variables used in the methodological plan of the survey, both at the sample level for the design variables, and at the respondent level for the variables involved in the treatment of non-response. Unfortunately, these variables are often not available in the final survey dataset, and the variance might therefore be miscalculated. Replication methods, which fully integrate the methodological plan applied into the survey, may be considered as good alternatives to this approach. This paper uses real data from the survey on “Racism and Ethno-racial Discrimination” carried out in Luxembourg in 2021, to compute and compare analytic variance estimators and bootstrap variance estimators. For this survey, both the linearization and the rescaled bootstrap lead to similar results, but the ultimate cluster variance estimator can be substantially biased. This suggests that the rescaled bootstrap may be a relevant approach.