Data Preprocessing and Data Leakage in Multivariate Time Series Forecasting

Data preprocessing is often treated as a routine step, although it can change forecasting accuracy considerably. This study measured the effect of five preprocessing stages (missing value imputation, anomaly detection, categorical encoding, normalization and smoothing) on ten machine learning, deep learning and statistical models using hourly multivariate Beijing PM2.5 data. Two or three common methods per stage were compared one at a time around a reference pipeline (17 configurations). Each preprocessing method learned its parameters from the training years 2010 to 2013 only, and in the 2014 test year it used only values observed before the forecast hour. The models forecast PM2.5 one hour ahead, and their errors were measured against the original measurements with MAE, RMSE and MAPE. The stages that modify the target series had the largest influence: smoothing the series raised the MAE of every model by 5% to 93%, and statistical outlier removal by up to 35%. Scoring against the preprocessed series, a form of data leakage, made the same forecasts appear better, with median MAE reductions of 4% to 55%. The effects of imputation and normalization varied by model family. Preprocessing should therefore be evaluated together with the model and against unmodified observations.