Do multi-agent LLMs improve causal identification? A controlled, cost-aware study on CLadder

Recent research has demonstrated that multi-agent collaboration frameworks can enhance causal reasoning performance in large language models. However, it remains unclear whether these benefits extend to small open-source language models (1B-15B parameters) and whether the performance gains justify the increased computational overhead compared to single-agent prompting strategies. This study presents a comparison of multi-agent collaboration against four chain-of-thought (CoT) prompting strategies—vanilla prompting, simple CoT, guided CoT, and self-consistency—on two causal reasoning benchmarks: CLadder and Corr2Cause. We evaluate small open-source models by systematically measuring task performance and deployment-relevant compute cost (per-query latency, total tokens consumed, and number of inference calls). Of 15 paired McNemar tests against the best paired-tested single-agent baseline per model (self-consistency could be included in this paired test for only 3 of 15 (benchmark, model) pairs; the remaining 12 rows compare against the best of vanilla, simple CoT, and guided CoT, because per-trajectory self-consistency logs were not retained), only 2 reach significance after Benjamini–Hochberg correction at q = 0.10 : one favors multi-agent (Gemma-3-12b on CLadder, Δ = + 4.80 pp), the other favors single-agent (Phi-4 on Corr2Cause, Δ = − 11.47 pp). The remaining 13 comparisons are statistically indistinguishable, with median absolute differences below 2 percentage points. At the same time, multi-agent runs require multiple sequential inference calls per question (minimum 3, worst-case 15 under our A = 3 , R = 4 configuration) versus exactly 1 for vanilla and Guided CoT; on the primary non-thinking models in Table 11 , multi-agent latency is 3.4 – 5.3 × that of Guided CoT and multi-agent output token volume is 4.2 – 4.6 × that of Guided CoT. These findings suggest that for SLM deployments, carefully designed single-agent prompting strategies offer a more favorable trade-off between performance and inference cost in causal reasoning tasks, at least under the same-model role-specialization multi-agent configuration evaluated here.