Informality features in AI-generated academic writing: A corpus-based comparison between human and AI

A comparison between AI-generated and human-written academic texts is crucial for assessing LLMs’ proficiency in academic writing and providing insights into academic writing instruction. While previous research has primarily focused on lexical, syntactic, and discoursal features, the investigation of writing style is relatively underdeveloped. The present study focused on a set of linguistic features, which are generally labeled as “informal” by style guides and writing manuals. We examined the differences between AI-generated and human-generated academic abstracts in terms of ten informal linguistic features. Our study was based on a large-scale dataset of scientific abstracts, consisting of comparable ChatGPT-generated and human-authored texts. The results showed that the use of informality features in AI-generated academic texts differs significantly from human-authored ones with small to moderate effect sizes. In addition, the writing style of AI is characterized by a consistent and standardized expression with less variability than that of human authors. We further underscore the similarities and disparities in the use of informality features between ChatGPT and human writers. By revealing AI’s strengths and limitations, these findings deepen our understanding of how AI reproduces or approximates the stylistic conventions expected in academic writing and offer valuable insights for integrating AI tools into writing instruction. • AI-generated abstracts differ from human-written ones in informality. • ChatGPT exhibits consistent writing style with less variability than humans. • We analyzed informality indices to compare AI and human academic writing. • Findings show similarities and disparities in informality between ChatGPT and humans.