When a Reranking Bonus Becomes a Language Filter: Score-Scale Mismatch in Reciprocal Rank Fusion for Multilingual RAG
Multilingual Retrieval-Augmented Generation (RAG) systems serving low-resource languages can suffer from unintended retrieval bias when language preferences are applied without accounting for the score scale of the underlying fusion method. This paper investigates a failure mode in a trilingual agricultural advisory system, Bhunaga, supporting English, Hindi, and Marathi. The system uses BGE-M3 dense and sparse retrieval combined through Reciprocal Rank Fusion (RRF) with a rank constant of k = 60. We demonstrate that an additive same-language bonus of +0.10 exceeds the maximum attainable fused score range of 0.0217 by approximately 4.6 times, effectively turning a soft reranking preference into a hard language filter. We derive the partition condition mathematically and evaluate three configurations: additive bonus, multiplicative bonus, and no language preference. Experiments on a corpus of 114 chunks across 24 documents show that the additive bonus reduces keyword recall by 45% for Hindi and 39% for Marathi compared with no bonus. English retrieval remains unchanged, acting as an internal control. We also identify a misleading monitoring metric: same-language retrieval rate reaches 1.00 precisely when retrieval quality is at its worst. The study proposes a scale-aware approach to score adjustment, recommends multiplicative adjustments where appropriate, and demonstrates why multilingual retrieval systems should evaluate cross-lingual retrieval quality rather than relying solely on same-language retrieval rates. Keywords: Multilingual RAG, Reciprocal Rank Fusion, Cross-Lingual Retrieval, Information Retrieval, BGE-M3, Score-Scale Mismatch, Natural Language Processing.
Authors
- Aishwarya Deshmukh
- Harshal Punjahari Agre
- Jayesh Vilas Kapade
- Manish Yadav
Institutions
- MIT Art, Design and Technology University (IN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24332200
- Primary Topic
- Information Retrieval and Search Behavior
- Type
- preprint