Query-Aware Context Compression for Efficient Long-Context Language Model Inference

Query-aware context compression can reduce the amount of source text forwarded to a language model, but keyword ranking may discard evidence connected through intermediate entities. We present QAB-R, a training-free sentence selector combining BM25, query-identifier-constrained anchor selection, canonicalized entity matching, and at most two bridge-linked candidates under a fixed sentence budget. We first evaluate a single-bridge baseline on 300 synthetic single- and two-hop queries. We then execute 300 additional held-out stress cases covering paraphrases, competing entity notes, identifier formatting variation, and combined changes; a separate 60-case test extends source contexts to 58,368 whitespace-delimited words. At a 10% sentence budget, QAB-R preserved complete two-hop support in all 60 cases of each stress condition, whereas single-bridge retention fell to 0% under competing notes and format changes. For 1,024 fact pairs, the median QAB-R CPU preprocessing time was 66.3 ms, and the selected text contained 5,741 of 58,368 input words. The study reports evidence completeness, retained input length, and CPU preprocessing cost; downstream reader accuracy and GPU inference speed are distinct system-level outcomes.

Authors

Institutions

Publication Details

Journal
Jisuanji shenghuojia.
Published
2026-11-13
DOI
https://doi.org/10.54097/zhed3474
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Query-Aware Context Compression for Efficient Long-Context Language Model Inference

Aaron McLeod, Maya Laurent
3 links
Jisuanji shenghuojia.
Topic Modeling
article

Query-Aware Context Compression for Efficient Long-Context Language Model Inference

Aaron McLeod, Maya Laurent
article en

Abstract

Query-aware context compression can reduce the amount of source text forwarded to a language model, but keyword ranking may discard evidence connected through intermediate entities. We present QAB-R, a training-free sentence selector combining BM25, query-identifier-constrained anchor selection, canonicalized entity matching, and at most two bridge-linked candidates under a fixed sentence budget. We first evaluate a single-bridge baseline on 300 synthetic single- and two-hop queries. We then execute 300 additional held-out stress cases covering paraphrases, competing entity notes, identifier formatting variation, and combined changes; a separate 60-case test extends source contexts to 58,368 whitespace-delimited words. At a 10% sentence budget, QAB-R preserved complete two-hop support in all 60 cases of each stress condition, whereas single-bridge retention fell to 0% under competing notes and format changes. For 1,024 fact pairs, the median QAB-R CPU preprocessing time was 66.3 ms, and the selected text contained 5,741 of 58,368 input words. The study reports evidence completeness, retained input length, and CPU preprocessing cost; downstream reader accuracy and GPU inference speed are distinct system-level outcomes.

Jisuanji shenghuojia.Vol. 15(2)
University of Waterloo (CA), University of Calgary (CA)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon