Reconstruction accuracy versus predictive performance: evaluating imputation methods under real-world missingness

Abstract Missing data are a common challenge in epidemiological research and may affect both statistical inference and predictive modelling. This study evaluated the impact of different imputation methods on downstream predictive performance and imputation accuracy using the National Health and Nutrition Examination Survey dataset with its inherent missingness patterns. Missing values in body mass index, fasting glucose, and total cholesterol were handled using five single-imputation methods—mean imputation, linear regression, K-nearest neighbour, random forest, and CatBoost—with complete case analysis included as a reference approach. Predictive performance was assessed using a logistic regression model for hypertension and evaluated by the area under the receiver operating characteristic curve, Cohen’s Kappa, sensitivity, and specificity across 300 repeated train–test splits. In addition, a masked validation framework was applied to assess reconstruction accuracy using root mean squared error and mean absolute error under both completely random and covariate-dependent masking at 20% and 50% missingness levels. The results showed that predictive performance was highly consistent across the single-imputation methods, with generally small differences in AUC and related metrics across repeated data splits. In contrast, model-based methods, particularly random forest and linear regression, achieved lower reconstruction error compared with mean imputation. However, these improvements in imputation accuracy did not translate into substantial improvements in predictive performance. These findings highlight a disconnect between imputation accuracy and downstream modelling outcomes, suggesting that more complex imputation methods do not necessarily improve predictive performance in this setting. The results emphasise the importance of aligning the choice of imputation method with the analytical objective and the data structure.