Where the Questions Come From: Prompt Provenance as a Bound on Generative-Search Visibility Measurement

Commercial products report how often generative search engines cite a business. Every such number is conditional on a question set, and that question set is a free parameter: the same brand, measured on the same platforms in the same week, can be assigned very different visibility depending on where its questions came from. We name this parameter prompt provenance and give it a three-class taxonomy — natively authored, machine-generated, and observed search queries. We report two experiments. In the first, a single brand is measured twice on the same five generative-search surfaces with the same replicate structure, differing only in corpus provenance, in four language markets (English, German, Spanish, French; 864 observations). The machine-generated corpus returns the higher citation rate in all four, but the size of the gap varies more than thirteenfold: −24.1 points in English (p = 5.9 × 10−4), −18.5 in Spanish (p = 0.009), −5.6 in German and −1.9 in French (both within noise). Pooled across the three non-English markets the difference is −8.6 points (p = 0.034). That variability is the finding, not a weakness of it. A practitioner cannot know in advance whether switching corpus will move their headline number by two points or by twenty-four, which makes the parameter's disclosure more necessary rather than less. The direction is informative too: the machine-generated corpus was the more flattering one every time. In the second, a leather-goods retailer is measured against the nearest available category corpus and returns 0 of 108 (95% CI 0–3%) — a precise-looking zero produced by a corpus that asks about winter coats and merino base layers. Re-measured against a purpose-built leather corpus it returns 0 of 90 (95% CI 0–4%), which is a real finding. The two zeros are statistically indistinguishable: identical instrument, comparable intervals, platforms answering normally. Only provenance separates the empty measurement from the informative one. We then show that in a census of 16 commercial products, none discloses the provenance of its prompts, and five do not disclose the prompt count at all. Our claim is not that these products are inaccurate. It is that a visibility figure without a stated provenance is not interpretable, that the effect is large enough to exceed the period-over-period changes such products report, and that disclosure costs nothing. Competing interest. The author builds and operates a product in the category this paper examines. All 1,062 observations and a dependency-free analysis script are deposited with the paper, so every figure can be recomputed rather than taken on trust.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-10
DOI
https://doi.org/10.5281/zenodo.22694994
Primary Topic
Expert finding and Q&A systems
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Where the Questions Come From: Prompt Provenance as a Bound on Generative-Search Visibility Measurement

Muhammad Arshad
Zenodo (CERN European Organization for Nuclear Research)
Expert finding and Q&A systems
preprint

Where the Questions Come From: Prompt Provenance as a Bound on Generative-Search Visibility Measurement

Muhammad Arshad
preprint en

Abstract

Commercial products report how often generative search engines cite a business. Every such number is conditional on a question set, and that question set is a free parameter: the same brand, measured on the same platforms in the same week, can be assigned very different visibility depending on where its questions came from. We name this parameter prompt provenance and give it a three-class taxonomy — natively authored, machine-generated, and observed search queries. We report two experiments. In the first, a single brand is measured twice on the same five generative-search surfaces with the same replicate structure, differing only in corpus provenance, in four language markets (English, German, Spanish, French; 864 observations). The machine-generated corpus returns the higher citation rate in all four, but the size of the gap varies more than thirteenfold: −24.1 points in English (p = 5.9 × 10−4), −18.5 in Spanish (p = 0.009), −5.6 in German and −1.9 in French (both within noise). Pooled across the three non-English markets the difference is −8.6 points (p = 0.034). That variability is the finding, not a weakness of it. A practitioner cannot know in advance whether switching corpus will move their headline number by two points or by twenty-four, which makes the parameter's disclosure more necessary rather than less. The direction is informative too: the machine-generated corpus was the more flattering one every time. In the second, a leather-goods retailer is measured against the nearest available category corpus and returns 0 of 108 (95% CI 0–3%) — a precise-looking zero produced by a corpus that asks about winter coats and merino base layers. Re-measured against a purpose-built leather corpus it returns 0 of 90 (95% CI 0–4%), which is a real finding. The two zeros are statistically indistinguishable: identical instrument, comparable intervals, platforms answering normally. Only provenance separates the empty measurement from the informative one. We then show that in a census of 16 commercial products, none discloses the provenance of its prompts, and five do not disclose the prompt count at all. Our claim is not that these products are inaccurate. It is that a visibility figure without a stated provenance is not interpretable, that the effect is large enough to exceed the period-over-period changes such products report, and that disclosure costs nothing. Competing interest. The author builds and operates a product in the category this paper examines. All 1,062 observations and a dependency-free analysis script are deposited with the paper, so every figure can be recomputed rather than taken on trust.

Zenodo (CERN European Organization for Nuclear Research)
Genetic Technologies (Australia) (AU)
Expert finding and Q&A systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.