Recommending Is Not Citing: Separable and Competing Mechanisms for Entity Mention and Source Attribution in Large Language Models
When a language model answers a question from numbered sources, two outputs that are often treated as one can come apart: whether it names an entity (recommends a brand) and whether it cites the source that supports that entity. We study this split in a controlled retrieval-augmented setting: five numbered sources, one per brand, a purchase question, and an explicit instruction to cite with bracketed numbers. The target brand's source sits at position [3] and is either neutral or carries one concrete fact. We read next-token probabilities in five open-weight models (Qwen3.5 4B, 27B, 122B-A10B, 397B-A17B-FP8 and DeepSeek-V4-Flash) and generated text in two API models (GPT-4.1-mini, Gemini 2.5 Flash). In every model, the concrete fact raises the probability of naming the brand (e.g. 1.7% to 47.7% in the 27B; 1.8% to 44.4% in the 397B) and lowers the probability of citing that same source once the brand is named (5.9% to 3.2%; 11.2% to 2.3%). In the API models the citation drop appears only under the citation instruction. Internally, difference-of-means directions for "recommend" and "cite" are nearly orthogonal at the positions where each decision is made (mean |cosine| 0.03–0.09, against split-half noise ceilings an order of magnitude higher) and partially anti-aligned at the position right after the brand. Adding the "recommend" direction raises naming and lowers citation beyond a random-direction control in three models. Activation patching places the citation decision in the last ~10% of depth, after the naming decision in the 27B and 397B. With sparse autoencoders on the 27B, ablating about ten features cuts the fact-driven recommendation from 70% to 32% (random ablation: 68%), whereas the late-layer features tied to citation look like reference-formatting features; we found no feature resembling "this claim needs support". Attention analysis splits citing into whether to open a bracket and which index to write: the index is correct 99% of the time and is computed redundantly (ablating up to 32 attention heads matches random ablation), while the fact acts on whether. A short observational sample of Google AI Overviews in Argentina is consistent with mention and attribution being separate in production. We conclude that entity mention and source attribution should be measured and studied as separate behaviours.
Authors
- Nicolás Bargioni (ORCID: https://orcid.org/0009-0000-6574-4455)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5281/zenodo.23938245
- Primary Topic
- Topic Modeling
- Type
- preprint