Causal Intervention Is What Large Language Models Need for Spatio-Temporal Forecasting

Spatio-temporal forecasting plays a crucial role in the dynamic perception of smart cities, such as traffic flow prediction, renewable energy forecasting, and load prediction. Its objective is to understand the patterns of spatio-temporal changes under the interaction of various factors. Accurate spatio-temporal forecasting relies on sufficient high-quality data and powerful models. However, in reality, data is often sparse. In such cases, while adaptive graphs and large language models (LLMs) can maintain performance, they face issues of spatial spurious associations and hallucinations, respectively. These issues hinder the ability of the model to learn and infer cross spatio-temporal and cross-scale features effectively. To address this, we propose a novel model termed spatio-temporal causal intervention large language model (STCInterLLM). This model employs a newly designed causal intervention encoder to update spatial spurious correlations in the spatio-temporal adaptive graph. Subsequently, the novel chain-of-action prompting text is utilized to enforce the decomposition of the prediction process, thereby enhancing the causal representation of features while mitigating hallucinations in LLMs. Finally, a lightweight marker alignment module ensures the consistency between the encoder, prompting text, and LLM, enabling accurate forecasting of distinct scale spatio-temporal evolution patterns. Extensive experiments conducted on power distribution systems integrated with renewable energy sources and transportation systems encompassing diverse types of data, demonstrate that the proposed STCInterLLM consistently achieves state-of-the-art performance across significantly varied scenarios. Codes are available at https://github.com/lishijie15/STCInterLLM.