Linguistic Analyses of Written Corrective Feedback for Chinese as a Second Language: ChatGPT Versus Human Teachers

ABSTRACT This study conducted linguistic analyses of the written corrective feedback (WCF) for Chinese as a second language (CSL) provided by chat generative pre‐trained transformer (ChatGPT) and human teachers (including preservice teachers and senior teachers). An assessment framework was proposed, composed of two main domains of ‘language accuracy’ and ‘content expressivity’, which have further sub‐indices associated with measurable linguistic parameters. Both qualitative and quantitative data analyses were conducted on a self‐constructed Vietnamese CSL writing sample corpus. The qualitative analysis identified four features of ChatGPT compared with human teachers: (1) more word order changes; (2) more formal expressions used; (3) unsatisfactory inferencing ability; and (4) insufficient literacy skills. The results of quantitative analyses indicated ChatGPT tended to make unnecessary revisions and adopt more difficult vocabulary. Our overall data indicate that the WCF provided by ChatGPT is especially good in language accuracy (though occasionally inadequate knowledge of the Chinese language). In the domain of content expressivity, ChatGPT tends to provide feedback in more difficult vocabularies, which does not fully understand the real needs of CSL students. Our analytic framework and empirical data led to insightful discussions on AI and human intelligence in the context of second language teaching and learning.