Recommending Is Not Citing: Separable and Competing Mechanisms for Entity Mention and Source Attribution in Large Language Models

When a language model answers a question from numbered sources, two outputs that are often treated as one can come apart: whether it names an entity (recommends a brand) and whether it cites the source that supports that entity. We study this split in a controlled retrieval-augmented setting: five numbered sources, one per brand, a purchase question, and an explicit instruction to cite with bracketed numbers. The target brand's source sits at position [3] and is either neutral or carries one concrete fact. We read next-token probabilities in five open-weight models (Qwen3.5 4B, 27B, 122B-A10B, 397B-A17B-FP8 and DeepSeek-V4-Flash) and generated text in two API models (GPT-4.1-mini, Gemini 2.5 Flash). In every model, the concrete fact raises the probability of naming the brand (e.g. 1.7% to 47.7% in the 27B; 1.8% to 44.4% in the 397B) and lowers the probability of citing that same source once the brand is named (5.9% to 3.2%; 11.2% to 2.3%). In the API models the citation drop appears only under the citation instruction. Internally, difference-of-means directions for "recommend" and "cite" are nearly orthogonal at the positions where each decision is made (mean |cosine| 0.03–0.09, against split-half noise ceilings an order of magnitude higher) and partially anti-aligned at the position right after the brand. Adding the "recommend" direction raises naming and lowers citation beyond a random-direction control in three models. Activation patching places the citation decision in the last ~10% of depth, after the naming decision in the 27B and 397B. With sparse autoencoders on the 27B, ablating about ten features cuts the fact-driven recommendation from 70% to 32% (random ablation: 68%), whereas the late-layer features tied to citation look like reference-formatting features; we found no feature resembling "this claim needs support". Attention analysis splits citing into whether to open a bracket and which index to write: the index is correct 99% of the time and is computed redundantly (ablating up to 32 attention heads matches random ablation), while the fact acts on whether. A short observational sample of Google AI Overviews in Argentina is consistent with mention and attribution being separate in production. We conclude that entity mention and source attribution should be measured and studied as separate behaviours.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-04
DOI
https://doi.org/10.5281/zenodo.23938245
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Recommending Is Not Citing: Separable and Competing Mechanisms for Entity Mention and Source Attribution in Large Language Models

Nicolás Bargioni
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Recommending Is Not Citing: Separable and Competing Mechanisms for Entity Mention and Source Attribution in Large Language Models

Nicolás Bargioni
preprint en

Abstract

When a language model answers a question from numbered sources, two outputs that are often treated as one can come apart: whether it names an entity (recommends a brand) and whether it cites the source that supports that entity. We study this split in a controlled retrieval-augmented setting: five numbered sources, one per brand, a purchase question, and an explicit instruction to cite with bracketed numbers. The target brand's source sits at position [3] and is either neutral or carries one concrete fact. We read next-token probabilities in five open-weight models (Qwen3.5 4B, 27B, 122B-A10B, 397B-A17B-FP8 and DeepSeek-V4-Flash) and generated text in two API models (GPT-4.1-mini, Gemini 2.5 Flash). In every model, the concrete fact raises the probability of naming the brand (e.g. 1.7% to 47.7% in the 27B; 1.8% to 44.4% in the 397B) and lowers the probability of citing that same source once the brand is named (5.9% to 3.2%; 11.2% to 2.3%). In the API models the citation drop appears only under the citation instruction. Internally, difference-of-means directions for "recommend" and "cite" are nearly orthogonal at the positions where each decision is made (mean |cosine| 0.03–0.09, against split-half noise ceilings an order of magnitude higher) and partially anti-aligned at the position right after the brand. Adding the "recommend" direction raises naming and lowers citation beyond a random-direction control in three models. Activation patching places the citation decision in the last ~10% of depth, after the naming decision in the 27B and 397B. With sparse autoencoders on the 27B, ablating about ten features cuts the fact-driven recommendation from 70% to 32% (random ablation: 68%), whereas the late-layer features tied to citation look like reference-formatting features; we found no feature resembling "this claim needs support". Attention analysis splits citing into whether to open a bracket and which index to write: the index is correct 99% of the time and is computed redundantly (ablating up to 32 attention heads matches random ablation), while the fact acts on whether. A short observational sample of Google AI Overviews in Argentina is consistent with mention and attribution being separate in production. We conclude that entity mention and source attribution should be measured and studied as separate behaviours.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon