Traffic speed prediction based on stacked denoising and lightweight Transformer

To address the challenges of noisy traffic speed data and the high computational complexity of existing Transformer models, this paper proposes a novel traffic speed prediction framework based on stacked denoising and a lightweight Transformer. First, a stacked denoising approach combining wavelet denoising and a denoising autoencoder is proposed to preprocess raw traffic data and obtain the denoised traffic data. Second, we propose a lightweight decoder architecture that replaces the complex multi-head cross-attention mechanism in the standard Transformer decoder with a novel module combining Sparse FeedForward Network (SFFN) and FeedForward Network (FFN), leading to significant parameter reduction while maintaining accuracy. Further, the combined loss function based on Huber loss and MAPE loss is proposed for training the model and accelerating model convergence. Finally, three evaluation metrics (MAE, RMSE, and MAPE) are used to evaluate the multi-step prediction performance on two datasets. Four evaluation metrics (Parameters, FLOPs, MACs and Latency) were employed to assess the model’s complexity. Compared to the baseline models, the proposed model exhibits notable advantages in computational efficiency while sustaining high prediction performance. Ablation studies confirm the necessity and effectiveness of each component. The results indicate that this framework offers a practical solution for intelligent transportation systems, balancing computational efficiency with reliable accuracy in real-world traffic speed prediction tasks.