Spatiotemporal memory probabilistic sparse transformer for traffic flow prediction

Traffic flow prediction is a key task in Intelligent Transportation Systems (ITS). Although existing approaches have achieved notable progress, most models still struggle to preserve and leverage historical patterns embedded in traffic nodes. This leads to limited ability in modeling periodic and trending features, thereby restricting their effectiveness in capturing complex and dynamic spatiotemporal dependencies. To address these challenges, we propose a novel prediction framework named Spatiotemporal Memory Probabilistic Sparse Transformer (SMPST). It is designed to enhance the modeling of long-term spatiotemporal dynamics while maintaining a balance between computational efficiency and generalization. SMPST integrates a hierarchical periodic temporal embedding and multi-scale temporal convolution to extract periodic and trending features across multiple time scales while reducing temporal redundancy. A memory-based spatial attention mechanism is used to store and retrieve representative historical traffic patterns, enhancing the model's perception of long-term flow regularities. Moreover, the integration of spatiotemporal convolution with probabilistic sparse self-attention facilitates multi-scale spatiotemporal feature interaction and reduces computational overhead. Experiments conducted on five real-world traffic flow datasets show that SMPST significantly outperforms state-of-the-art baselines, particularly in long-term forecasting, achieving superior robustness and accuracy.