Source Use and Cultural Image Questions: Small External-Label Pilots in Yoruba and Swahili
Multilingual evaluation can confound question comprehension, answer language, supplied evidence, and cultural knowledge. We report two small pilots that separate some of these factors using published human labels. In AfriQA, 24 questions, equally divided between Yoruba and Swahili, are each evaluated with a native question and English passage, the published English question and the same passage, and a native question with no passage. Every condition explicitly requires an English source span or JSON null. In a separate Afri-MCQA study, 12 Yoruba image groups are tested with native and English question-option pairs, plus a native-language condition without the image. Two local model configurations complete 234 requests, including 18 interface checks excluded from locked results. Native-question source exact matches are 3/12 Yoruba and 7/12 Swahili for Qwen3.5, and 7/12 and 6/12 for Gemma4; English-question differences are mixed. Both models obey all 24 empty-passage controls. Cultural-image accuracy is 5/12 and 7/12, compared with 2/12 and 3/12 without images. These descriptive results support a reproducible evaluation interface and show why schema compliance, extraction accuracy, and cultural image use should be reported separately. They do not establish native-language service quality or new community validation.
Authors
- Saurav Keshari Aryal (ORCID: https://orcid.org/0000-0001-9815-9295)
Publication Details
- Journal
- Preprints.org
- Published
- 2026-11-03
- DOI
- https://doi.org/10.20944/preprints202611.0210.v1
- Primary Topic
- Topic Modeling
- Type
- preprint