Do Weights and Complex Design Matter in Structural Equation Modeling for Social Studies? An Example of the Political Satisfaction-Political Trust Model

The analyses applying complex design and weighting techniques in Structural Equation Modeling, which encompass both theoretical considerations, practical limitations and possibilities within available statistical software, still appear to be limited — despite the topic not being new. Using the European Social Survey (ESS) and Multigroup Confirmatory Factor Analysis (MGCFA) on political satisfaction and political trust, the estimation consequences of four complex sample analysis scenarios are demonstrated. These include a naïve approach and those adjusted with sampling weights and design variables using two software packages: Mplus8.4 and Stata18. Based on the literature review, only a few studies have included detailed information about the weighting approach in the analyses performed. The absence of this information could affect the accuracy of results and the replication of estimations, as shown in this paper. Empirically, it is demonstrated that assuming a simple random sample in MGCFA can result in divergent coefficients, biased estimates of latent covariances and mean differences, and underestimated standard errors. These in turn affect confidence intervals, goodness-of-fit indices, composite reliability, and convergent validity metrics, potentially undermining the validity of the research design and results, the reliability of inferences, and the comparability of outcomes across countries. Additionally, it was illustrated how results vary when using different software packages under similar estimation settings. In conclusion, there is an urgent need for all researchers who apply complex sample analysis in their work, using weights and design variables, to clearly present their applied approach to analyses, depending on the software used.