Evaluation of PM2.5 prediction performance of CMAQ and AI models (LSTM and Transformer) in an operational air quality forecasting system

• A framework (LSTM and Transformer) to overcome the limitations of CMAQ is proposed. • LSTM achieved minimal bias and the highest F1-score in the Seoul Metropolitan Area, enhancing forecast reliability. • The Transformer model significantly enhanced the ability to detect high concentration PM 2.5 events. • AI models enable more accurate and reliable real-time air quality forecasts. Accurate forecasting of surface PM 2.5 concentrations is critical for effective air quality management and timely decision making. Traditional numerical models such as CMAQ often exhibit biases and errors due to uncertainties in inputs, limiting their predictive reliability. This study compared CMAQ with two AI-based models (LSTM and Transformer) across South Korea (19 regions) and the Seoul Metropolitan Area (SMA). Results showed that CMAQ suffered from substantial bias, large errors, and high false alarm rates, whereas the AI models consistently outperformed it. The LSTM model achieved stable accuracy across all statistical and categorical indicators, with an F1-score of 79% and nearly zero bias in the SMA. The Transformer model excelled in detecting high pollution events, with PODs of 79% nationwide and 94% in the SMA, and the lowest errors under “Bad” and “Very Bad” conditions. These complementary strengths highlight the potential of AI-based models to enhance operational PM 2.5 forecasting.