Granularity Masking: When Paper-Level Evaluation Misses Passage-Level Degradation
This study examines when paper-level retrieval metrics mask passage-level degradation. Across 1,309 QASPER questions, a controlled within-paper reversal preserved every paper ranking while reducing mean passage nDCG@10 by 0.175. Paper-level metrics detected the fallback and index-shift interventions in aggregate, although masking persisted for individual questions.
Authors
- S. M. Abdulla Al Mamun
Institutions
- University of Manchester (GB)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.5281/zenodo.24368103
- Primary Topic
- Information Retrieval and Search Behavior
- Type
- preprint