Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts

Large Language Models (LLMs) have demonstrated significant effectiveness across various NLP tasks, including text ranking. This study assesses the performance of large language models (LLMs) in listwise reranking for limited-resource African languages. We compare proprietary models RankGPT3.5, Rank4o-mini, RankGPTo1-mini and RankClaude-sonnet in cross-lingual contexts. Results indicate that these LLMs significantly outperform traditional baseline methods such as BM25-DT in most evaluation metrics, particularly in nDCG@10 and MRR@100. These findings highlight the potential of LLMs in enhancing reranking tasks for low-resource languages and offer insights into cost-effective solutions.

Authors

Institutions

Publication Details

Published
2024-12-27
DOI
https://doi.org/10.1109/icairc64177.2024.10900179
Citations
1
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.35
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts

Yixian Shen, Yiyi Tao, Shaoshuai Du, Yanxin Shen et al.
1 citations
Topic Modeling
0.35
article

Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts

Yixian Shen, Yiyi Tao, Shaoshuai Du, Yanxin Shen, Hang Zhang, Lun Wang, Chuanqi Shi
article en
1 citations

Abstract

Large Language Models (LLMs) have demonstrated significant effectiveness across various NLP tasks, including text ranking. This study assesses the performance of large language models (LLMs) in listwise reranking for limited-resource African languages. We compare proprietary models RankGPT3.5, Rank4o-mini, RankGPTo1-mini and RankClaude-sonnet in cross-lingual contexts. Results indicate that these LLMs significantly outperform traditional baseline methods such as BM25-DT in most evaluation metrics, particularly in nDCG@10 and MRR@100. These findings highlight the potential of LLMs in enhancing reranking tasks for low-resource languages and offer insights into cost-effective solutions.

Johns Hopkins University (US), Duke University (US), Simon Fraser University (CA), University of California San Diego (US), Amsterdam University of the Arts (NL), University of Amsterdam (NL)
Openalex Percentile: Top 29%
Topic Modeling
0.35
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.