A Comparative Study of Generative AI and Human Texts: Limits and Possibilities of Academic Writing

This study examines how the use of generative AI tools influences the content and style of university students’ writing and explores its educational implications. Despite the rapid spread of generative AI such as ChatGPT in higher education, empirical comparisons of its actual impact on student writing remain limited. To address this gap, 69 term papers were collected from the liberal arts course the ‘Philosophy of Life and Ecology in the 21st Century’ at D. University in Seoul, covering two cohorts: 2021 (without AI) and 2025 (with AI). A mixed-methods approach was adopted: text-mining techniques were employed to quantitatively compare sentence length, lexical diversity (TTR, MATTR), conjunction use, and sentiment scores, while the course instructor qualitatively reviewed rhetorical features, writing style, and reference practices.The analysis showed that AI-assisted papers were shorter and more concise, with lower lexical diversity than those written without AI. While overall conjunction frequency and variety revealed no significant difference, AI-assisted writing tended to rely more on illustrative and emphatic connectors rather than conditional ones. Sentiment scores were largely similar across both groups; however, from the instructor’s perspective, AI-assisted writing often reflected uniformity of thought and style, sometimes including fabricated references, which revealed the issue of AI hallucination. At the same time, a few students demonstrated critical engagement by explicitly reporting how and why they used AI tools.These findings suggest that prohibiting AI use in academic writing is insufficient. Instead, pedagogical design and assessment criteria should guide students to employ AI critically and creatively while ensuring fact-checking and personal interpretation. This study offers empirical evidence on the opportunities and challenges of AI-assisted writing, providing a foundation for balanced discussions in higher education.