Exploring EFL Writing Assessment in Omani Higher Education: A Comparative Study of Large Language Models and Human Raters

This study investigates the effectiveness of three automated assessors—ChatGPT, DeepSeek, and Gemini—in evaluating English as a Foreign Language (EFL) writing tasks across four foundation levels. The design was quantitative and comparative. It compared human ratings with AI-generated scores for two types of writing tasks. The evaluation was based on a rubric covering Task Achievement, Coherence and Cohesion, Grammar, and Lexical Resource. The participants were 36 EFL learners enrolled in the General Foundation Program at the University of Technology and Applied Sciences, Oman. Writing scores were analyzed using SPSS to identify differences between human and AI writing assessments. Results indicate that ChatGPT and DeepSeek closely align with human ratings at lower proficiency levels (Levels 1 and 2). However, discrepancies emerged at Level 4, especially in Coherence and Cohesion, Grammatical Range and Accuracy, and Lexical Resource. Gemini showed limited reliability and inconsistent alignment across all levels. The findings suggest that automated tools can support the assessment of simpler writing tasks. However, human raters remain essential for evaluating more complex language use. The study recommends optimizing AI scoring systems to enhance their ability to handle advanced language features and ensure fairness in high-stakes writing assessments.