Citing Without the Key: How LLMs Inherit Human Citation Practices

In 2003, Simkin and Roychowdhury estimated from repeated citation errors that only about 20% of citers read the original work they cite. Large language models now get the reference itself right in about 99% of cases. We ask whether they also cite the way humans do. Taking ten concepts of citation from the social sciences and humanities, we examined 123 texts written by Claude Sonnet 5.5 and ChatGPT, mostly lectures of about 3,000 words, with around 3,600 citations. In four respects the models behave like human authors: they remember the pagination of specific print editions, quote from memory (3 of 165 quotations traced to the cited edition were paraphrases dressed up as quotations), pay ritual citations to the big names of a topic, and, in Sonnet’s case, hedge where the source itself is cautious. In four others they do not: they confuse critics with allies, stretch sources (about 10% of Sonnet’s and 6% of ChatGPT’s specific claims are not in the cited work), argue against straw men, and end three lectures in four with a reconciling synthesis. And when the same work returns in another text, only 25–29% of the claims return with it. Borrowing Goffman’s notion of a key, we argue that the machine has inherited human citation practices but not the signals that tell a reader what a citation is doing. A shorter, edited version of this essay was published as: Kholin, E. (2026). The problem with AI citations is that they are all too human. LSE Impact Blog, 17 November 2026. https://blogs.lse.ac.uk/impactofsocialsciences/2026/11/17/the-problem-with-ai-citations-is-that-they-are-all-too-human/ Files. citing_without_the_key_preprint.pdf: the preprint, 6 pages, 5 figures. citing_without_the_key_data.zip: the 123 generated texts with their prompts, the extracted citations, the judgments, the results and the analysis scripts; see README.md inside. Passages of the cited books used as evidence are not included, since they are copyrighted.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24322555
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Citing Without the Key: How LLMs Inherit Human Citation Practices

Egor Kholin
3 links
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

Citing Without the Key: How LLMs Inherit Human Citation Practices

Egor Kholin
preprint en

Abstract

In 2003, Simkin and Roychowdhury estimated from repeated citation errors that only about 20% of citers read the original work they cite. Large language models now get the reference itself right in about 99% of cases. We ask whether they also cite the way humans do. Taking ten concepts of citation from the social sciences and humanities, we examined 123 texts written by Claude Sonnet 5.5 and ChatGPT, mostly lectures of about 3,000 words, with around 3,600 citations. In four respects the models behave like human authors: they remember the pagination of specific print editions, quote from memory (3 of 165 quotations traced to the cited edition were paraphrases dressed up as quotations), pay ritual citations to the big names of a topic, and, in Sonnet’s case, hedge where the source itself is cautious. In four others they do not: they confuse critics with allies, stretch sources (about 10% of Sonnet’s and 6% of ChatGPT’s specific claims are not in the cited work), argue against straw men, and end three lectures in four with a reconciling synthesis. And when the same work returns in another text, only 25–29% of the claims return with it. Borrowing Goffman’s notion of a key, we argue that the machine has inherited human citation practices but not the signals that tell a reader what a citation is doing. A shorter, edited version of this essay was published as: Kholin, E. (2026). The problem with AI citations is that they are all too human. LSE Impact Blog, 17 November 2026. https://blogs.lse.ac.uk/impactofsocialsciences/2026/11/17/the-problem-with-ai-citations-is-that-they-are-all-too-human/ Files. citing_without_the_key_preprint.pdf: the preprint, 6 pages, 5 figures. citing_without_the_key_data.zip: the 123 generated texts with their prompts, the extracted citations, the judgments, the results and the analysis scripts; see README.md inside. Passages of the cited books used as evidence are not included, since they are copyrighted.

Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon