Performance of different algorithms in dealing with missing data in the presence of structural bias: a simulation study

Abstract Background Missing data can compromise causal effect estimation by introducing bias, reducing precision, and distorting statistical uncertainty. In addition to complete case analysis (CCA), we evaluated three commonly used imputation methods in R: multiple imputation by chained equations (mice); additive regression, bootstrapping, and predictive mean matching (aregImpute); and single imputation by chained random forests (missRanger). Methods We simulated two datasets under three missing mechanisms: missing completely at random (MCAR), missing at random (MAR), and missing not at random mechanisms (MNAR). For each scenario, we generated three missing proportions (10%, 25%, 50%) and four sample sizes ( n = 300, 600, 1200, 5000). We estimated effects using the three imputation methods and CCA. Performance was evaluated using relative bias and root-mean-square error on the log odds-ratio scale, confidence-interval coverage and width, standard-error calibration, and computation time. Results Bias and root mean squared error (RMSE) increased with higher missingness and decreased with larger sample sizes. CCA and mice generally showed lower bias and RMSE than aregImpute and missRanger. aregImpute achieved the highest coverage overall but had wider CIs and higher RMSE. missRanger performed worse than the other methods. Conclusion Although no missing-data handling strategy consistently outperformed others, mice provided more balanced performance across the evaluated scenarios. These findings suggest that researchers should carefully consider bias-variance trade-offs when choosing an imputation strategy when dealing with missing data in the presence of structural bias.