Source Use and Cultural Image Questions: Small External-Label Pilots in Yoruba and Swahili

Multilingual evaluation can confound question comprehension, answer language, supplied evidence, and cultural knowledge. We report two small pilots that separate some of these factors using published human labels. In AfriQA, 24 questions, equally divided between Yoruba and Swahili, are each evaluated with a native question and English passage, the published English question and the same passage, and a native question with no passage. Every condition explicitly requires an English source span or JSON null. In a separate Afri-MCQA study, 12 Yoruba image groups are tested with native and English question-option pairs, plus a native-language condition without the image. Two local model configurations complete 234 requests, including 18 interface checks excluded from locked results. Native-question source exact matches are 3/12 Yoruba and 7/12 Swahili for Qwen3.5, and 7/12 and 6/12 for Gemma4; English-question differences are mixed. Both models obey all 24 empty-passage controls. Cultural-image accuracy is 5/12 and 7/12, compared with 2/12 and 3/12 without images. These descriptive results support a reproducible evaluation interface and show why schema compliance, extraction accuracy, and cultural image use should be reported separately. They do not establish native-language service quality or new community validation.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-03
DOI
https://doi.org/10.20944/preprints202611.0210.v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Source Use and Cultural Image Questions: Small External-Label Pilots in Yoruba and Swahili

Saurav Keshari Aryal
Preprints.org
Topic Modeling
preprint

Source Use and Cultural Image Questions: Small External-Label Pilots in Yoruba and Swahili

Saurav Keshari Aryal
preprint en

Abstract

Multilingual evaluation can confound question comprehension, answer language, supplied evidence, and cultural knowledge. We report two small pilots that separate some of these factors using published human labels. In AfriQA, 24 questions, equally divided between Yoruba and Swahili, are each evaluated with a native question and English passage, the published English question and the same passage, and a native question with no passage. Every condition explicitly requires an English source span or JSON null. In a separate Afri-MCQA study, 12 Yoruba image groups are tested with native and English question-option pairs, plus a native-language condition without the image. Two local model configurations complete 234 requests, including 18 interface checks excluded from locked results. Native-question source exact matches are 3/12 Yoruba and 7/12 Swahili for Qwen3.5, and 7/12 and 6/12 for Gemma4; English-question differences are mixed. Both models obey all 24 empty-passage controls. Cultural-image accuracy is 5/12 and 7/12, compared with 2/12 and 3/12 without images. These descriptive results support a reproducible evaluation interface and show why schema compliance, extraction accuracy, and cultural image use should be reported separately. They do not establish native-language service quality or new community validation.

Preprints.org
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Source Use and Cultural Image Questions: Small External-Label Pilots in Yoruba and Swahili — Saurav Keshari Aryal · Preprints.org (2026) | TGRS Research Map | TGRS