A linguistic comparison between ChatGPT-generated and nonnative student-generated short story adaptations: a stylometric approach

Abstract The present study provides a qualitative-quantitative linguistic analysis of AI-generated versus human-generated short story adaptations with the aim of detecting the stylometric features that distinguish each style. The author analyzed 15 classic short story adaptations written by nonnative ESL students in an Egyptian university and compared them to 15 adaptations of the same short stories generated by the AI bot, ChatGPT, in terms of content as well as the stylometric language features of length, lexical choices and word frequencies. The results show that the AI-generated text was more loyal to the main theme, plot line and character description of the original story but used language that was more complex, descriptive, unique and bias-free, whereas the student-generated text deviated from the original story in terms of theme, plot, characters and context and used language structures that were simpler and more repetitive. The non-native features characterizing student-generated text included long sentences with excessive use of coordinators, basic vocabulary, frequent intensifiers and L1-induced structures and sociocultural stereotypes. The implications of the study can be used to inform the ESL community of ways to distinguish ChatGPT-generated from nonnative human-generated linguistic features with the aim of attributing texts to their rightful authors as well as finding out the linguistic and stylistic features that characterize non-native English text.