Evaluating Human–AI Rating Agreement in Foundation EFL Writing: Evidence From Analytic Rubric Scoring in Kuwait

Abstract Artificial intelligence (AI) is increasingly used to support writing assessment, yet it remains unclear how closely AI-generated scores correspond with human judgment in Foundation English as a foreign language (EFL) contexts. This study examined agreement between AI-generated and human ratings of Foundation EFL writing in Kuwait. The dataset comprised 692 writing assessment records from 162 students enrolled in ENGL097 and ENGL098. Overall scores and six analytic criteria (topic sentence, body and development, conclusion, organization, vocabulary, and grammar) were analyzed using descriptive statistics, Pearson and Spearman correlations, intraclass correlation coefficients (ICCs), mean differences, error indices, and Bland–Altman analysis. AI scores showed very high agreement with the average of two human raters (r = .992; absolute-agreement ICC = .991). The mean difference was 0.24 points on a 50-point scale, the mean absolute error was 0.89 points, and 93.1% of AI ratings fell within two points of the human average. Criterion-level ICCs ranged from .953 to .965, with vocabulary lowest and conclusion highest. A significant proportional bias indicated that disagreement varied across the score range. The findings suggest that AI scoring can provide consistent support for Foundation EFL writing assessment when applied through a clearly defined analytic rubric, although human oversight remains essential.