GTFormer: A Geospatial Temporal Transformer for Crowd Flow Prediction

Crowd flow prediction is an essential task on human mobility, and the prediction of the origin-destination (OD) matrix has drawn strong interest in recent years. A conventional approach has been to capture temporal dependencies from spatial features, extracted by a graphical or a spatial convolution. It requires a well-trained spatial feature extractor, which is difficult to obtain in the presence of dynamic and compound spatio-temporal dependencies. We propose Geospatial Temporal transFormer (GTFormer), built as a stack of identical spatio-temporal transformer units, which alleviates the need for pre-training a feature extractor and simplifies the training process. While each unit processes the geospatial and temporal dependencies independently and linearly combines their outputs, stacking such units enables GTFormer to capture compound dependencies. Furthermore, we address the computational burden of training transformers with novel self-attention (SA) mechanisms for geospatial and temporal modules of the transformer units, respectively. Our empirical study on shared bikes and taxi flow prediction shows the advantage of GTFormer over the current state-of-the-arts. We also present an ablation study and visual analyses to evaluate the contributions of individual components to performance and interpretability.