MSTDP: Multimodal Spatio-Temporal Forecasting with Self-Correcting Attention and Hierarchical Feature Interaction

Real-world spatiotemporal data are frequently imbalanced across spatial regions and collected from heterogeneous sources with different structures, sampling frequencies, and semantics. Sparse regions provide limited supervision, whereas modality redundancy and structural heterogeneity make it difficult to exploit complementary information during fusion. To address these challenges, we propose MSTDP, a Multimodal Spatio-Temporal Data Prediction framework. A causal target-clock alignment procedure first maps high-frequency, low-frequency, and irregular observations to the forecasting timeline without using future information. MSTDP then employs modality-specific encoders to obtain source-dependent representations and constructs first-, second-, and third-order interaction features through a Multilayer Feature Interaction module. A Hierarchical Feature Fusion module aggregates the interaction features. To improve intermediate and high-level representations, we further introduce two plug-and-play components: a Unified Feature Interaction Module, which performs soft alignment and bidirectional feature exchange between heterogeneous token sequences, and a Self-Correcting Transformer Module, which recalibrates modality-specific and fused representations through cross-modal attention. A spatial-unit-balanced training objective first normalizes the prediction error within each region and then averages equally across regions, preventing data-rich regions from dominating the gradients. We evaluate MSTDP on four real-world benchmarks covering urban housing-price forecasting, county-level crop-yield prediction, and traffic-speed forecasting. Relative to the strongest baseline in each setting, MSTDP reduces RMSE by up to 8.6% and MAPE by up to 11.5%. Ablation studies confirm that hierarchical fusion, unified feature interaction, and self-correction provide complementary improvements. Observation-density stress tests further show that the model remains robust when entire spatial units are unavailable. These results demonstrate the effectiveness of MSTDP on the four evaluated datasets and provide empirical evidence that its interaction and correction modules can be reused across the studied housing, agriculture, and traffic forecasting tasks. Broader transferability to other multisource spatiotemporal problems remains to be established.