Interpretable linguistic features for multi-trait prediction of english essay scores, and what they do not license a feedback tool to claim

Automated writing assessment is increasingly proposed for language-learning tools, but the feedback offered is often loosely connected to what the underlying model supports. This study examines multi-trait prediction of English essay scores from interpretable linguistic features, and asks what a feedback tool built on that model may legitimately claim. Working with the released 3,911-essay training partition of the ELLIPSE Corpus of English-language-learner writing, whose six corpus-provided score variables (cohesion, syntax, vocabulary, phraseology, grammar, conventions) are scored 1.0–5.0, we computed 77 linguistic features and evaluated models with nested cross-validation (5 folds × 3 repeats), compared using the Nadeau–Bengio corrected resampled t -test with Holm–Bonferroni adjustment. Because fold dependence leaves an effective 3.16 independent folds, effect sizes are mean differences with corrected intervals, fold win rates and essay-level probabilities of superiority rather than standardised d z . Ridge regression over these features achieved a mean columnwise RMSE of 0.5271, a 19.3% reduction against a constant-mean baseline, with no detectable difference from TF-IDF representations. Four further findings emerged. First, dimension-specific modelling outperformed a holistic model, but against a base-rate-ranked constant strategy neither single- nor two-variable identification of a writer’s weaker areas was distinguishable from the baseline, and at two the baseline was ahead (0.476 against 0.463); what survives is graded—nominated variables sit 0.188 standard deviations below the writer’s own typical standing in 15 of 15 folds. Second, an ordinal Frank–Hall formulation outperformed interval regression in 15 of 15 folds. Third, regularised linear coefficients proved unsafe for feedback under correlated features, two carrying signs opposite their marginal associations. Fourth, the marginal associations underlying the feedback are overwhelmingly general-proficiency signal—9 of 78 survive control for general score level—so the feedback is exploratory rather than diagnostic. Against the corpus’s 44 true prompt labels, prompt-level optimism is +0.0039 MCRMSE (95% CI [0.0016, 0.0068]), too small to affect any conclusion. The prototype’s uncertainty displays are empirical error bands—well calibrated marginally but not across the score range—not prediction intervals. Corpus inter-rater agreement is modest and no validation evidence accompanies it, so we report prediction of recorded scores, not measurement of writing ability.