Beyond Document Relevance: Evaluating Citation-Set Structure in Citation Recommendation

When authors of a scientific paper cite several papers at a single point, they are often not aiming to cite the exact same claim repeatedly. Citation sets include a high degree of author judgement — they are often chosen together to cover a conceptual space, reference both a problem and its main methods, or pair a method with a result it is usually compared to. Despite this, citation recommendation, as it is usually posed, ignores this set-level structure, and instead scores each target in isolation to maximize its independent semantic similarity to the citing sentence. We show that in the case of recommending a citation set, this may be the wrong approach. We introduce LOCUS, a controlled evaluation that determines whether or not a retriever can tell which of a paper’s own citations belong at one sentence rather than another one in the text, and that looks beyond the use of topic-level similarity. A simple, training-free co-citation reranker built from past citation structure improves Mean Reciprocal Rank (MRR) by +0.104 to +0.124 across four retrievers. The gain largely comes from 61.3% of cases where the candidate and an already-known citation are linked in the citation graph, while using citations from other locations in the same paper hurts performance. In corpus-wide retrieval with a self-retrieved first citation, SPECTER2 still improves Recall@20 by +0.075. Further, once one citation is known, the retriever is 2.6× more effective at recovering the remaining members at Recall@1.

Authors

Institutions

Publication Details

Journal
Valency Hub
Published
2026-11-16
DOI
https://doi.org/10.83295/hh2e-9r2x
Primary Topic
Information Retrieval and Search Behavior
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Beyond Document Relevance: Evaluating Citation-Set Structure in Citation Recommendation

Karthik Ram, J. S. Bloom, Makan Farhoodimoghadam, Avery Louis
1 links
Valency Hub
Information Retrieval and Search Behavior
preprint

Beyond Document Relevance: Evaluating Citation-Set Structure in Citation Recommendation

Karthik Ram, J. S. Bloom, Makan Farhoodimoghadam, Avery Louis
preprint en

Abstract

When authors of a scientific paper cite several papers at a single point, they are often not aiming to cite the exact same claim repeatedly. Citation sets include a high degree of author judgement — they are often chosen together to cover a conceptual space, reference both a problem and its main methods, or pair a method with a result it is usually compared to. Despite this, citation recommendation, as it is usually posed, ignores this set-level structure, and instead scores each target in isolation to maximize its independent semantic similarity to the citing sentence. We show that in the case of recommending a citation set, this may be the wrong approach. We introduce LOCUS, a controlled evaluation that determines whether or not a retriever can tell which of a paper’s own citations belong at one sentence rather than another one in the text, and that looks beyond the use of topic-level similarity. A simple, training-free co-citation reranker built from past citation structure improves Mean Reciprocal Rank (MRR) by +0.104 to +0.124 across four retrievers. The gain largely comes from 61.3% of cases where the candidate and an already-known citation are linked in the citation graph, while using citations from other locations in the same paper hurts performance. In corpus-wide retrieval with a self-retrieved first citation, SPECTER2 still improves Recall@20 by +0.075. Further, once one citation is known, the retriever is 2.6× more effective at recovering the remaining members at Recall@1.

Valency Hub
University of California, Davis (US), University of California, Berkeley (US), Stanford University (US)
Information Retrieval and Search Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon