Beyond Document Relevance: Evaluating Citation-Set Structure in Citation Recommendation
When authors of a scientific paper cite several papers at a single point, they are often not aiming to cite the exact same claim repeatedly. Citation sets include a high degree of author judgement — they are often chosen together to cover a conceptual space, reference both a problem and its main methods, or pair a method with a result it is usually compared to. Despite this, citation recommendation, as it is usually posed, ignores this set-level structure, and instead scores each target in isolation to maximize its independent semantic similarity to the citing sentence. We show that in the case of recommending a citation set, this may be the wrong approach. We introduce LOCUS, a controlled evaluation that determines whether or not a retriever can tell which of a paper’s own citations belong at one sentence rather than another one in the text, and that looks beyond the use of topic-level similarity. A simple, training-free co-citation reranker built from past citation structure improves Mean Reciprocal Rank (MRR) by +0.104 to +0.124 across four retrievers. The gain largely comes from 61.3% of cases where the candidate and an already-known citation are linked in the citation graph, while using citations from other locations in the same paper hurts performance. In corpus-wide retrieval with a self-retrieved first citation, SPECTER2 still improves Recall@20 by +0.075. Further, once one citation is known, the retriever is 2.6× more effective at recovering the remaining members at Recall@1.
Authors
- Karthik Ram (ORCID: https://orcid.org/0000-0002-0233-1757)
- J. S. Bloom (ORCID: https://orcid.org/0000-0002-7777-216X)
- Makan Farhoodimoghadam (ORCID: https://orcid.org/0009-0007-3982-3791)
- Avery Louis (ORCID: https://orcid.org/0009-0009-0176-2326)
Institutions
- University of California, Davis (US)
- University of California, Berkeley (US)
- Stanford University (US)
Publication Details
- Journal
- Valency Hub
- Published
- 2026-11-16
- DOI
- https://doi.org/10.83295/hh2e-9r2x
- Primary Topic
- Information Retrieval and Search Behavior
- Type
- preprint