StrucFormer: Structural Prior Guided Transformer for Mobile Crowdsensing Data Inference

The inherent constraint of the "human-in-the-loop" sensing mechanism, imposes mobile crowdsensing with high dynamics and uncertainty, ultimately leading to the issue of incomplete data collection. Current data inference solutions in mobile crowdsensing can be broadly categorized as low-rank models and deep learning models. Low-rank models apply structural prior for data inference, but have limited model capacity, while deep learning models possess salient feature expressivity, but are prone to overfitting in sparse crowdsensing scenarios. In this paper, we try to absorb the strengths of both two paradigms, and propose a structural prior guided Transformer, StrucFormer, for crowd-sensing data inference. Specifically, we exploit structural prior of low-rankness to power canonical Transformer from the aspects of input embedding, attention forming and model regularization, which enables the model to precisely capture the spatiotemporal and multi-type data correlations for accurate inference with only sparse observations. Extensive empirical results demonstrate the superiority of StrucFormer in terms of accuracy and generality in heterogeneous urban sensing tasks. The code is available at: https://github.com/CUPK-K/StrucFormer.