DQN-SCI: A Reinforcement Learning Method for Sequential Causal Inference

Causal inference is crucial in decision-making as it helps estimate the effects of interventions or treatments on key variables. Currently, the task of causal inference focuses on estimating causal effects using fully measured features. However, this does not accurately reflect real-world scenarios, where the challenge often involves first selecting the relevant features to measure before performing the estimation. To solve the challenge, in this paper, we introduce a new task called sequential causal inference and propose an innovative approach named deep Q-network for sequential causal inference (DQN-SCI). DQN-SCI designs a ‘decider-inferencer’ framework to solve the task, where the decider first selects valuable features for the subsequent inferencer. DQN-SCI outperforms the compared methods on a synthetic dataset, demonstrating its effectiveness.