When the sandwich makes you hesitate, replicate: on sampling variance estimation of multilevel models under complex sample design

Abstract Large-scale surveys routinely rely on complex sample designs, necessitating special consideration of sampling variance estimation in multilevel models (MLM). While the sandwich estimator is widely used for this purpose, its implementation, particularly regarding stratification and weighting, remains challenging. Alternatively, the lesser-known replication methods provide a valid alternative; but they are often misunderstood as being only suitable for single-level models and are not widely supported by software packages. This paper clarifies key aspects of implementing both methods under two-level MLM common in large-scale surveys. We provide practical guidance on incorporating sample weights, correctly identifying variance strata for sandwich estimation, and applying replication-based variance estimation in MLM. Two simulation studies evaluate the performance of each method under correct and incorrect specifications, including omission of informative level-1 weights. Results demonstrate that the sandwich estimator and replication methods yield comparable variance estimates when implemented correctly and highlight the consequences of common misapplications. An empirical example using TIMSS 2015 Australia data is used to illustrate these issues in practice. This work contributes to improved methodological soundness in multilevel modeling and calls for expanded software support for replication methods in MLM.