The impact of automated writing evaluation on writing gains

Abstract This study examines the effectiveness of ChatGPT in improving English writing skills for English as a foreign language (EFL) students over a nine-week intervention. Specifically, it explores ChatGPT’s impact on syntactic and lexical complexity—key dimensions of writing development. The study compares two groups: one receiving teacher feedback alone and another using ChatGPT alongside teacher feedback. Participants included 105 first-year university students in Belgium, divided into these two groups. Results indicate that while ChatGPT significantly affects text length and complexity, it does not necessarily lead to sustained writing improvement. This finding is crucial for researchers and educators integrating artificial intelligence (AI) tools with traditional teaching, as it refines current insights into generative AI’s educational potential. The study underscores the need for balanced approaches in writing instruction, where AI complements rather than replaces human feedback, ensuring that short-term benefits translate into long-term writing proficiency for EFL learners.