A Calibrated Multi-LLM Consensus Framework for Reducing Model-Specific Scoring Asymmetry in EFL Writing Assessment

Large language models (LLMs) are increasingly deployed in second-language (L2) writing assessment. However, substantial disparities persist in how different architectures operationalize shared scoring criteria. This study investigates model-specific assessment asymmetry across ChatGPT-4o, Gemini-2.5 Pro, and DeepSeek-V3, and evaluates whether a calibrated multi-LLM consensus framework can mitigate these scoring discrepancies. Forty argumentative essays composed by Chinese EFL learners were independently evaluated by the three models using a standardized prompt and an analytic rubric encompassing task relevance, clarity of ideas, coherence, and accuracy, benchmarked against human mean scores. Individual models and ensemble configurations were systematically validated across 100 iterations of five-fold nested cross-validation. Cross-model comparisons revealed that inter-model divergence varied significantly by dimension, exhibiting pronounced asymmetry in holistic ratings and task relevance compared to clarity of ideas, coherence, and accuracy. The full consensus framework—integrating model-specific calibration, error-based weighting, and meta-calibration—yielded an RMSE of 1.160 and an MAE of 0.889, relative to 1.144 and 0.874 for unweighted averaging. The paired RMSE difference of 0.016 (95% case-cluster bootstrap CI [−0.114, 0.149]) indicated no statistically significant advantage in overall predictive accuracy over equal weighting. Nevertheless, the calibrated framework achieved superior bias mitigation, evidenced by a reduced absolute mean bias and a narrower median-split residual gap. Supplementary analyses further delineated direct inter-model discrepancies and proficiency-dependent residual bias. These findings demonstrate that model-specific evaluative asymmetries transcend simple mean-level shifts, and that while calibration-driven consensus enhances bias regulation across score bands, this error control does not automatically translate into improved global predictive precision.