Dynamic data reconciliation for enhancing the prediction performance of long short-term memory network

Abstract In modern process industries, long short-term memory (LSTM) network is widely used for data-driven modeling. Constrained by measuring instruments and environments, the measured datasets are generally with Gaussian/non-Gaussian distributed measurement noise. The noisy datasets will impact the modeling accuracy of the LSTM network and decrease the prediction performance of it. Aiming at addressing prediction performance impairment of the LSTM network under noisy datasets with Gaussian/non-Gaussian distribution, this study introduces dynamic data reconciliation (DDR) both into LSTM network training and into LSTM network test. Results show that DDR improves not only the data quality based on noisy datasets and the training outputs via the Bayesian formula in the model training step, but also the prediction performance based on offline measured information and the test outputs. The implementation scheme of DDR for Gaussian and non-Gaussian distributed noise is purposely designed. The effectiveness of DDR on the LSTM model is verified in a numerical example and a case involving a set of shared wind power datasets.