Writing with machines:How AI assistance shapes linguistic structures

Large language models (LLMs) such as ChatGPT are increasingly integrated into everyday writing. While prior studies have examined their effects on text quality and fluency (e.g., Dhillon et al., 2024), little is known about how machine assistance shapes the underlying linguistic structures of human-authored text. Here, we conducted a writing experiment using a GPT-4o-powered interface, adapted from Lee et al. (2022). We recruited 166 native speakers of American English and randomly assigned them to one of<br/>six writing conditions. These conditions were designed to simulate a range of human-AI interactions, from unaided writing to light-touch assistance to near co-authorship (see Table 1).<br/>We ask two questions:<br/>1. How much influence does machine assistance have on writing?<br/>To measure this, we tracked the origin of each word in the final text – human-generated, AI-generated, or jointly edited. As shown in Table 2, the local suggestion mode, Phrase Continuation, yielded the highest level of human-AI interaction, while Machine Completion led to the greatest presence of synthetic language. These findings indicate that AI assistance strongly shapes the surface composition of human-authored texts (Lee et al., 2022; Padmakumar &amp; He, 2024).<br/>2. How much structural difference does machine assistance create?<br/>Even if AI contributes a substantial portion of the final text, does this affect the types of linguistic structures used? Analyses of lexical, semantic, syntactic, and stylistic similarity showed that naturally written and AI-assisted texts were structurally alike, suggesting that while machine assistance affects who produces what, it does not radically alter the inventory of linguistic structures in the output. Together, these findings suggest that LLMs can substantially shape the surface composition of writing without fundamentally altering its structural profile. This points to a distinction between authorship and structure in human-AI co-writing, with implications for debates on linguistic diversity and authorship in AI-mediated communication.