STReLU: Spatio–Temporal Rectified Linear Unit for Traffic Forecasting

Spatio-temporal neural networks have achieved remarkable success in traffic forecasting by modeling complex temporal dynamics and spatial correlations. These advances are largely due to the combination of essential linear transformations and nonlinear activations. While prior networks have considered the heterogeneous information across nodes and time steps in their linear transformations, they typically apply the same nonlinear activation function to all data. This approach treats representations from different nodes and times as i.i.d. random variables, filtering irrelevant information uniformly and ignoring spatio-temporal heterogeneity. To address this limitation, we propose a novel Spatio-Temporal-aware Rectified Linear Unit (STReLU) as a nonlinear activation function for traffic forecasting. STReLU adaptively constructs the parameters of the activation function by incorporating spatio-temporal information, enabling dynamic filtering of irrelevant information for different inputs. The parameters of STReLU are generated by a hyperfunction that leverages spatial attention to capture node-wise correlations and a timestamp lookup to extract temporal patterns. Extensive experiments on four real-world traffic datasets demonstrate that our STReLU achieves state-of-the-art performance. Specifically, STReLU achieves an improvement of 2.3% in MAE, 1.9% in RMSE, and 1.1% in MAPE compared to the best-performing baselines on PeMSD3, PeMSD4, PeMSD7, and PeMSD8 datasets.