Answering the fan-out does not predict AI Overview citation: a pre-registered test across 235 queries
AI search systems answer a question by running several related searches, a technique Google calls query fan-out. A market of tools now scores pages by how many of these sub-queries they answer, on the premise that coverage raises the chance of being cited. This study tested that premise with a pre-registered observational design. The sample was 235 non-branded keywords for which a page from one of 46 domains had been cited in a Google AI Overview. Candidate pages were the organic top ten plus every cited page. Fan-out was observed directly from Gemini with Google Search grounding (705 calls) and Claude with web search (470 calls), and guessed by a model asked to predict it (470 calls). Results pages were collected for 539 sub-queries. In conditional logistic regression within each query (158 test queries, 1,190 pages, 670 cited), a better organic rank for the head query was strongly associated with citation (odds ratio 0.43 per standard deviation of log rank) and longer pages were cited more (1.30). Heading similarity to the query, ranking in the top ten for a sub-query, and the share of sub-queries a page answers added no out-of-sample discrimination (change in within-query AUC from minus 0.003 to plus 0.004, against a pre-set bar of plus 0.02), and guessed sub-queries did as well as observed ones. Observed fan-out was small: 2.35 clusters of sub-queries per keyword against 16.1 guessed, and Gemini ran no search at all in 47% of its answers. Of cited pages outside the head query's top ten, 14% ranked in the top ten for any observed sub-query. A same-day replication on a second data provider agreed. Fan-out coverage scores are not supported as predictors of AI Overview citation; rank for the head query and length are.
Authors
- Piers Butler
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-03
- DOI
- https://doi.org/10.5281/zenodo.23931636
- Primary Topic
- Information Retrieval and Search Behavior
- Type
- preprint