MaXIFE: Multilingual and Cross-lingual Instruction Following Evaluation

With the rapid adoption of large language models (LLMs) in natural language processing, the ability to follow instructions has emerged as a key metric for evaluating their practical utility.However, existing evaluation methods often focus on single-language scenarios, overlooking the challenges and differences present in multilingual and cross-lingual contexts.To address this gap, we introduce MaXIFE: a comprehensive evaluation benchmark designed to assess instruction-following capabilities across 23 different languages with 1667 verifiable instruction tasks.MaXIFE integrates both Rule-Based Evaluation and Model-Based Evaluation, ensuring a balance of efficiency and accuracy.We applied MaXIFE to evaluate several leading commercial LLMs, establishing baseline results for future comparisons.By providing a standardized tool for multilingual instructionfollowing evaluation, MaXIFE aims to advance research and development in natural language processing.

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.acl-long.698
Primary Topic
Second Language Learning and Teaching
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

MaXIFE: Multilingual and Cross-lingual Instruction Following Evaluation

Yile Liu, Jinglu Hu, Xiu Jiang, Liang Li et al.
Second Language Learning and Teaching
article

MaXIFE: Multilingual and Cross-lingual Instruction Following Evaluation

Yile Liu, Jinglu Hu, Xiu Jiang, Liang Li, Ziwei Ma, ChangJing ChangJing
article en

Abstract

With the rapid adoption of large language models (LLMs) in natural language processing, the ability to follow instructions has emerged as a key metric for evaluating their practical utility.However, existing evaluation methods often focus on single-language scenarios, overlooking the challenges and differences present in multilingual and cross-lingual contexts.To address this gap, we introduce MaXIFE: a comprehensive evaluation benchmark designed to assess instruction-following capabilities across 23 different languages with 1667 verifiable instruction tasks.MaXIFE integrates both Rule-Based Evaluation and Model-Based Evaluation, ensuring a balance of efficiency and accuracy.We applied MaXIFE to evaluate several leading commercial LLMs, establishing baseline results for future comparisons.By providing a standardized tool for multilingual instructionfollowing evaluation, MaXIFE aims to advance research and development in natural language processing.

Quality Education
Openalex Percentile: Top 79%
Second Language Learning and Teaching
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.