Causal feature selection using a conditional independence test with a balanced trade-off between Type I and Type II errors

Purpose Causal feature selection focuses on identifying the Markov Blanket (MB) of the class attribute, providing an optimal solution for feature selection. Most current causal feature selection algorithms rely on conditional independence (CI) tests to identify MBs. However, traditional CI tests result in an imbalanced trade-off between Type I and Type II errors, limiting their adaptability and overall effectiveness. Therefore, this paper aims to propose a CI test that achieves a balanced trade-off between Type I and Type II errors, improving the performance of causal feature selection algorithms. Design/methodology/approach In this paper, the authors propose adaptive scaling down (ASD) G2-test for causal feature selection. Specifically, the ASD G2-test dynamically adjusts the G2 statistic by considering the sample size, variable distributions, and other data characteristics, thereby achieving a balanced trade-off between Type I and Type II errors. Findings Experiments show that the ASD G2-test significantly outperforms traditional CI tests. On benchmark datasets, the ASD G2-test improved F1 scores by 0.4–1.1% (vs classical G2-test) across sample sizes of 1000, 2000 and 5,000. On real-world datasets, the ASD G2-test increased accuracy by 1.15–1.40% (vs classical G2-test) and 6.16–6.68% (vs χ2-test) for RF, DT and ANN classifiers. Originality/value Most existing studies do not consider the balance between Type I and Type II errors when using CI tests in causal feature selection algorithms. The ASD G2-test introduces a quantifiable formula to achieve a balanced trade-off Type I and Type II errors. Using the ASD G2-test in causal feature selection algorithms can achieve better performance.