Detecting model misfit in structural equation modeling with machine learning—a proof of concept

Despite the popularity of structural equation modeling in psychological research,accurately evaluating the fit of these models to data is still challenging. Using fixed fitindex cutoffs is error-prone due to the fit indices’ dependence on various features of themodel and data (“nuisance parameters”). Nonetheless, applied researchers mostly rely onfixed fit index cutoffs, neglecting the risk of falsely accepting (or rejecting) their model.With the goal of developing a broadly applicable method that is almost independent ofnuisance parameters, we introduce a machine learning (ML)-based approach to evaluatethe fit of multi-factorial measurement models. We trained an ML model based on 173model and data features that we extracted from 1,323,866 simulated data sets and modelsfitted by means of confirmatory factor analysis (i.e., training observations). We evaluatedthe performance of the ML model based on 1,659,386 test observations unseen duringmodel training. The ML model performed very well in detecting model (mis-)fit in themajority of conditions, thereby outperforming the fixed fit index cutoffs by Hu and Bentler(1999) across the board. Only minor misspecifications—single neglected residualcorrelations (and cross-loadings), in particular—proved to be challenging to detect. Fromthis proof-of-concept study we conclude that ML is very promising in the context of modelfit evaluation.