Automated written corrective feedback of generative AI in EFL students’ English argumentative essays: A systematic analysis of Claude 3.5 Sonnet

Generative AI has demonstrated promising potential in automated written corrective feedback. This study aimed to evaluate Claude 3.5 Sonnet's error feedback compared to human raters and explore the potential reasons for error feedback discrepancies based on EFL students’ argumentative essays. The major findings are as follows: (1) Claude 3.5 Sonnet exhibited a strong capability to provide error feedback, with an overall precision of 94%, recall of 89%, and accuracy of 95%. It was most effective in detecting and correcting run-on sentences, as well as errors related to capitalization, spelling and subject-verb agreement. However, discrepancies existed between Claude 3.5 Sonnet and human raters, particularly for pronoun, sentence structure, and word choice errors. (2) Compared to professional human raters, Claude 3.5 Sonnet had limited semantic competence and was less sensitive to essay fluency. Therefore, generative AI should be used as a supplementary tool for automated written corrective feedback rather than a substitute for human raters.