How Reproducible Is AI Search Visibility? Prompt-Paraphrase Sensitivity and a Confidence-Aware Measurement Protocol for Generative Engine Optimization
Generative Engine Optimization (GEO) is now sold as a measurable service, yet its standard measurement is a single point estimate: run a fixed prompt basket through a model, count brand mentions, report a percentage. We ask whether that number is reproducible, using three controlled experiments on Russian-language queries. (A) Across four dates, the mean pairwise Jaccard overlap of cited domains was 0.356 (33 queries, openai:gpt-4o-mini-search-preview). (B) Across five back-to-back runs within ten minutes, it was 0.722 (8 queries, perplexity:sonar); only 49 of 95 query-domain pairs (51.6%) survived all five runs. (C) Our main result isolates a variance source the market ignores: prompt phrasing. In an 8 intents x 5 paraphrases x 5 repeats design (200 answers), we define a Prompt-Paraphrase Index PPI = W - B, with W the within-phrasing reproducibility and B the between-phrasing similarity. We measured W = 0.723, B = 0.431, PPI = 0.292 (cluster bootstrap 95% CI [0.206, 0.372]; permutation test, Holm-corrected max p = 0.0008). Of 300 intent-domain pairs, 173 (58%) had a presence range of at least 50 percentage points across paraphrases, and for 35% at least one phrasing cited a domain in all five runs while another cited it in none. The vendor who writes the prompt basket controls this variance, so an unaudited GEO percentage is not a stable property of a website. We then propose the GEO-Signal Passport, operational metrics that replace the single percentage: Confidence-Adjusted Visibility (a Jeffreys lower bound), prompt-basket decomposition, a three-axis stability report, citation half-life, a citation/absorption/fidelity split, and a difference-in-differences design for attribution. All formulas are numerically verified; code and data are released for replication.
Authors
- Stanislav Kirichenko (ORCID: https://orcid.org/0009-0001-2914-9541)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-07-22
- DOI
- https://doi.org/10.5281/zenodo.21498261
- Primary Topic
- Advanced Multi-Objective Optimization Algorithms
- Type
- preprint