Improving Data Quality Completeness using Deep Neural Network (DNN) Prediction

Data quality serves as an indicator for transforming information into high-quality outputs. This indicator varies, and the Data Management Association (DAMA) has identified six primary factors, including completeness. Completeness relates to missing values, which are a common issue in time series data. This study proposes addressing the missing value and unknown value problems using Deep Neural Networks, specifically Recurrent Neural Networks (RNNs). While RNNs possess the capability to retain information, they are susceptible to the vanishing gradient problem. This challenge is mitigated through the introduction of Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), both of which demonstrate lower mean square error, indicating superior prediction and imputation accuracy compared to the mean imputation strategy.