To Err Is Human: How People Infer Authorship in the Age of AI

As large language models (LLMs) increasingly generate human-like text, it is becoming an increasingly pressing empirical and societal question: how can AI-generated text be distinguished from human-written text? Through two studies, this paper explored whether and how people judge AI authorship. In Study 1, participants (n = 101) judged the written responses to an inside joke prompt (the prompt chosen based on LLMs' response) and assessed each response on a 7-point scale of −3, definitely AI-written, to +3, definitely human-written, followed by self-reports of justifications. People's classifications were near chance accuracy (51.1%), but their continuous ratings did separate the human-written from the AI-written responses. Multivariate logistic regression showed that accuracy was best predicted by attention to surface-level errors and structural features rather than by deeper experiential cues (for example, interpretations of humor and personal schema matches). In Study 2, a new sample (n = 100) evaluated AI responses deliberately modified with spelling and grammatical errors. Accuracy dropped steeply in participants below chance (39.2%), validating the conclusion of Study 1. Together, the results of these two studies reveal that people's capacity to detect human expression rests less on detecting genuine lived experience than on superficial linguistic cues that are easy to manipulate.

Authors

Publication Details

Journal
Lecture Notes in Education Psychology and Public Media
Published
2026-11-17
DOI
https://doi.org/10.54254/2753-7048/2027.38988
Primary Topic
Authorship Attribution and Profiling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

To Err Is Human: How People Infer Authorship in the Age of AI

Yijia Xu
5 links
Lecture Notes in Education Psychology and Public Media
Authorship Attribution and Profiling
article

To Err Is Human: How People Infer Authorship in the Age of AI

Yijia Xu
article en

Abstract

As large language models (LLMs) increasingly generate human-like text, it is becoming an increasingly pressing empirical and societal question: how can AI-generated text be distinguished from human-written text? Through two studies, this paper explored whether and how people judge AI authorship. In Study 1, participants (n = 101) judged the written responses to an inside joke prompt (the prompt chosen based on LLMs' response) and assessed each response on a 7-point scale of −3, definitely AI-written, to +3, definitely human-written, followed by self-reports of justifications. People's classifications were near chance accuracy (51.1%), but their continuous ratings did separate the human-written from the AI-written responses. Multivariate logistic regression showed that accuracy was best predicted by attention to surface-level errors and structural features rather than by deeper experiential cues (for example, interpretations of humor and personal schema matches). In Study 2, a new sample (n = 100) evaluated AI responses deliberately modified with spelling and grammatical errors. Accuracy dropped steeply in participants below chance (39.2%), validating the conclusion of Study 1. Together, the results of these two studies reveal that people's capacity to detect human expression rests less on detecting genuine lived experience than on superficial linguistic cues that are easy to manipulate.

Lecture Notes in Education Psychology and Public MediaVol. 161(1)
Authorship Attribution and Profiling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon