A 50-prompt audit of six AI answer engines: citation volume, source mix and run-to-run instability

Generative answer engines do not rank pages, they select a handful to cite. We report a first-party citation log: one fixed set of fifty prompts, asked to six engines in English and Chinese, with every source the answer itself displayed recorded. Three findings: engines differ more from each other than languages do (independent sites take 80% of Doubao's citations and 6% of 360 AI's); citation volume spans more than threefold (16.4 per answer for Perplexity, 4.7 for Bing Copilot); and two runs of the same prompt share only part of their sources (Jaccard 0.48 for Doubao, 0.41 for Metaso, while Perplexity is sticky within a session at 0.66 but changes almost entirely across sessions at 0.16). ChatGPT is covered through third-party measurements rather than our own, and the engines we could not complete are reported instead of estimated.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-19
DOI
https://doi.org/10.5281/zenodo.24374260
Primary Topic
Information Retrieval and Search Behavior
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

A 50-prompt audit of six AI answer engines: citation volume, source mix and run-to-run instability

Jask Don
Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
preprint

A 50-prompt audit of six AI answer engines: citation volume, source mix and run-to-run instability

Jask Don
preprint en

Abstract

Generative answer engines do not rank pages, they select a handful to cite. We report a first-party citation log: one fixed set of fifty prompts, asked to six engines in English and Chinese, with every source the answer itself displayed recorded. Three findings: engines differ more from each other than languages do (independent sites take 80% of Doubao's citations and 6% of 360 AI's); citation volume spans more than threefold (16.4 per answer for Perplexity, 4.7 for Bing Copilot); and two runs of the same prompt share only part of their sources (Jaccard 0.48 for Doubao, 0.41 for Metaso, while Perplexity is sticky within a session at 0.66 but changes almost entirely across sessions at 0.16). ChatGPT is covered through third-party measurements rather than our own, and the engines we could not complete are reported instead of estimated.

Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon