CEET: Time-Series Casual Effect Estimation via Transformer Based Representation Learning

Accurately estimating causal effects in time series holds significant value across various fields, including healthcare and finance. Existing estimation methods often perform poorly under complex original distributions and struggle to capture the intricate long-range dependencies among confounding factors. To address these issues, this paper proposes a Transformer-based representation learning model for time series causal effect estimation, named Causal Effect Estimation Transformer (CEET). The model is built on the Transformer architecture and employs an encoder-decoder framework. The encoder captures complex dependencies in historical data and generates balanced representations using Multi-Head Self-Attention. The decoder leverages these representations, combined with Self-Attention and Cross-Attention mechanisms, to predict counterfactual outcomes for causal effect estimation. Additionally, the model incorporates relative position encoding to enhance its ability to capture long-range dependencies in sequences. Furthermore, CEET utilizes a generative adversarial network to filter out confounding information, thereby reducing the influence of confounding factors and further improving the model’s performance. Experimental results demonstrate that the proposed model achieves superior performance on multiple benchmark datasets, validating its effectiveness.