Air particulate matter (PM2.5) concentration prediction in Kuwait using vision transformer

Globally, air pollution has become one of the most concerning problems that affects the quality of human life. In Kuwait, one of the most prominent air pollutants is particulate matter 2.5 (PM2.5), which is a mixture of many inhalable particles that can cause various health issues. Predicting PM2.5 concentration levels will hopefully urge the decision makers to start taking actions against this concerning problem. Since deep learning has proven its efficiency in forecasting tasks, this study aims to develop a robust deep learning model for forecasting PM2.5 future concentrations in Kuwait. We utilized Kuwait air quality and weather data spanning the years 2017 to 2024. The time-series data was initially processed by handling missing values using three distinct data imputation techniques including Self-Attention-based Imputation for Time-Series (SAITS), Conditional Score-based Diffusion model for Imputation (CSDI), and linear interpolation. Furthermore, dimensionality reduction was implemented on the imputed datasets using principal component analysis (PCA), reducing the number of features from 10 to 4. Each of the imputed datasets was presented in time domain, frequency domain, and as principal components. Two data visualization techniques were employed including line graph images, to represent time domain data and principal components, and spectrograms that represent frequency domain data. Images typically possess rich visual patterns, allowing for extracting valuable features and hence, enhancing forecasting results. These images were then fed to a hybrid model that consists of a vision transformer (ViT) and Bidirectional Long Short-Term Memory (BiLSTM), where ViT was used to extract meaningful features from the input images and BiLSTM served to process these features sequentially and forecast PM2.5 concentration levels. The results revealed outstanding performance of our model, achieving the lowest Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE) values of 0.0191, 0.0031, and 0.0562, respectively. To assess the generalizability of our model, it was tested against two open-source datasets collected in Northern Ireland and Beijing in China. Evaluation results demonstrated significant reductions in the error-based metrics when compared with previous work.