The AI Answer Evidence Index: preregistration. Do the numbers in AI answers have sources?
This record registers the design and the instrument of The AI Answer Evidence Index before its main data collection. The study asks whether a reader who follows an AI answer engine's own sources can find there the figures the answer gives: do the numbers in AI answers have sources? Fifty buyer questions about prices, costs, fees and value were drawn by a written rule and a seed fixed beforehand from 100 questions taken verbatim from Google autocomplete suggestions (hl=en, gl=us) on 22 September 2026, across six sectors. No question was written by hand and none was picked by preference: each draw was made by a rule fixed in code beforehand. Google AI Overviews, ChatGPT and Gemini will each be asked every question once through their free public web interfaces, from a New York network exit, by fixed scripts: no AI model asks a question or reads a page during collection, and an operator may pass a verification page by hand, which every attempt records. Ten questions are asked a second time on a later day as a stability measure. A local model lists the figures in each answer; every listed figure is checked by a script against the pages the answer cites and gets one of six outcomes, and an unknown scores zero and stays in the denominator. The figures that are not found are classified, and a 40-figure sample is read by a second model; both readings are model readings and are reported as such. The deposit carries the method note (METHOD-NOTE.md, the registered text), the owner's dated decisions, the question frame with its selection rule, and the instrument: collection scripts, parsers, the page fetcher with its tests, the figure extractor, the check, the classifier, the validation sample and the summary scripts. MANIFEST.sha256 lists the SHA-256 hash of every registered file. The SHA-256 of MANIFEST.sha256 itself is 6146ee82ce380fb9350958d8bf90b5e1079afe8f87726ea2577a29ccab509c16. Every file of the deposit is open from the moment of registration, with no embargo and no restricted file. This is the second study in the series that began with the AI Search Evidence Index (doi:10.5281/zenodo.22257563). The results, the data and the code will be published as a separate record that references this registration.
Authors
- Minel Güneşoğlu
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-05
- DOI
- https://doi.org/10.5281/zenodo.23145556
- Primary Topic
- Information Retrieval and Search Behavior
- Type
- article
- Field-Weighted Citation Impact
- 0.00