Imputation of incomplete medical data using missing neighborhood perturbation denoising autoencoder

The problem of missing data in the clinical environment is a common one. However, the presence of these missing values or incorrect processing can affect downstream tasks. Imputation is the most effective way to solve the missing problem. Combining deep generative models and multiple imputation idea provides new ideas for the imputation of data, but some methods do not apply to mixed-type tabular data such as clinical data. The missing types of tabular data in real-world scenarios are varied and unpredictable, so it is necessary to construct a generalized imputation method. In this paper, we propose a multiple imputation architecture for missing neighborhood perturbation denoising autoencoder (MI_NPDAE) for imputing mixed types of incomplete medical tabular data for different missing mechanisms. In MI_NPDAE, the neighborhood information of the data is searched to find the best set of donors to be used as the input layer of the network, and the learning of the neighborhood features of the missing locations is ensured by reconstructing new inputs, while the addition of additive noise exposes the model to different missing mechanisms. The experiments were respectively conducted on a public dataset Breast and one lung cancer nutrition dataset from the Chinese Anti-Cancer Society. The experiments on the datasets show that the proposed model shows better results under different missing mechanisms and can maintain a low error on the comparison experiments on data with different missing ratios. In addition, the imputed results also show significant advantages in downstream prediction tasks.