Spatio-temporal epidemic forecasting with graph-based transformer

Abstract Epidemic forecasting plays a vital role in modern public health. The COVID-19 outbreak underscored the critical need for accurate and responsive models. Recent spatio-temporal Graph Neural Network (GNN) models that integrate human mobility networks face challenges in fully capturing complex, non-linear temporal dynamics and long-range spatial dependencies. To bridge this gap, we introduce two novel spatio-temporal architectures that combine GNNs with Transformer-based temporal modeling: a local-attention-model, which restricts self-attention to temporally adjacent windows of the same node, and a global-attention-model, which leverages full sequence-wide attention across all nodes to capture long-range dependencies. We benchmark our approaches against Persistence, Graph Convolutional Recurrent Network (GCRN) and Graph WaveNet baselines using two real-world datasets from Spain and Brazil. Our models show competitive and superior performance across most metrics compared to recurrent and temporal convolution baselines. The Linear Temporal Graph Convolutional Network (LinearTGCN) variant achieves the best Symmetric Mean Absolute Percentage Error (SMAPE) 24.74% and Mean Directional Accuracy (MDA) 72.52% on Spain dataset, outperforming the full attention-models. While, in Top-40 cities subset of Brazil dataset, local-attention-model slightly matches or outperforms the compared baselines with RMSE (3873.63) and SMAPE (83.47%). Our experiments demonstrate that simple linear models can match or exceed Transformers on structured time series, while Transformers show a great performance on noise or unstructured datasets like Brazil dataset. We found that SMAPE values varies across models by only a few percentage points, while the models are significantly better at directional prediction than the Persistence baseline.