Causal-aware training with counterfactual supervision and structural causal models improves the robustness of LLM-based vulnerability detection

Large language models (LLMs) fine-tuned for vulnerability detection achieve impressive performance on standard benchmarks, yet growing evidence suggests they exploit spurious correlations—such as code style, comment presence, or identifier naming conventions—rather than learning genuine vulnerability root causes. Recent rigorous evaluations have revealed that models reporting F1-scores above 90% on Big-Vul collapse to near-random performance on properly deduplicated datasets, with F1 dropping by up to 34 \(\times \) . This paper introduces CausalVul , a causal-reasoning framework that addresses this fundamental limitation through four synergistic contributions: (1) Counterfactual Vulnerability Analysis, which identifies minimal code perturbations that flip vulnerability labels to isolate root causes from confounders; (2) Structural Causal Model (SCM) Construction over 25 code features using the PC algorithm with chi-square independence tests, from which we derive adjusted association scores —observational associations adjusted for the observed feature set rather than interventional effects—used to rank and inject features; (3) Causal-Aware Model Training, combining a CodeBERT encoder with LoRA adaptation, contrastive learning on counterfactual pairs, and multi-task CWE classification; and (4) Mechanism-Grounded Accuracy, a novel evaluation metric validated against CVE patches. Under a rigorous project-level group-aware evaluation protocol, CausalVul achieves an F1-score of 76.4% on Devign—an improvement of +18.0 points over its own fine-tuned CodeBERT backbone under identical conditions, and +4.8 points over the strongest cross-backbone baseline (UniXcoder, 71.6%)—with an AUC-ROC of 0.862 and a Mechanism-Grounded Accuracy (the joint accuracy of the detection and CWE heads) of 54.8%. Matched non-causal controls indicate that approximately 9–10 of the 18.0 points are attributable to the causal selection of features and counterfactual edits, with the remainder coming from generic training improvements and from the presence of extra features and pairs. Under combined semantics-preserving lexical perturbations, CausalVul exhibits only a 5.1% F1 drop compared to 12.1–22.3% for baselines, a 2.4 \(\times \) improvement indicating reduced reliance on lexical shortcuts; we do not claim mechanism-level robustness, which would require control-flow- and data-flow-altering rewrites that we leave to future work. On the strict PrimeVul pair-level evaluation, all methods—including ours—perform poorly, a finding we report transparently to characterize the limits of current causal-aware training.