Knowledge-Distilled Multi-Task Model With Enhanced Transformer and Bidirectional Mamba2 for Air Quality Forecasting

Accurate air quality forecasting is crucial for public health and environmental policy, yet existing deep learning models often suffer from high computational costs and inadequate modeling of temporal dependencies and multi-pollutant interactions. This paper proposes a novel deep learning framework that integrates an Enhanced Transformer with Bidirectional Mamba2, optimized through <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-task learning</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">knowledge distillation</i>. Using a teacher–student paradigm, the teacher model captures rich temporal semantics, while the lightweight student model retains predictive accuracy with reduced inference costs. Key innovations include a hybrid architecture combining multi-scale global-local attention and long-range dependency modeling, a regression-specific knowledge distillation approach with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">soft target smoothing</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">intermediate representation transfer</i>, and an end-to-end multi-task design for joint forecasting of multiple pollutants. Extensive experiments on real-world datasets from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Guangzhou, Chengdu</i>, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Beijing</i> (2018–2022) across four key pollutants (PM2.5, PM10, NO<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub>, SO<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub>) demonstrate that our model significantly outperforms baseline methods, with the student model maintaining accuracy within 5% of the teacher while requiring fewer parameters. These results highlight the framework’s potential for accurate, scalable, and efficient air quality forecasting in real-time and resource-constrained environments.