Can AI reason from inside a tradition? A preregistered benchmark for tradition fidelity in language models

Language models are increasingly asked to speak for cultures, religions and other ways of knowing, and recent work suggests they often adopt a tradition's voice while reasoning from a narrower default. A 2026 preprint describes a "persona illusion" on ethical dilemmas rooted in Indigenous contexts; Agarwal and colleagues find that an average American is a closer match to Indian cultural values than any Indian-trained model they tested, and that prompting does not meaningfully improve alignment. Whether this gap can be closed by architecture, and not only by instructions, is untested. We propose the WhatIfWe Open Benchmark, which compares three conditions on problems where traditions are known to reason differently: tradition-grounded agents, whose retrieval is partitioned by tradition and whose claims must cite their sources; a single model prompted to answer as each tradition in turn; and a single model answering for all traditions at once. Practitioners formed in each tradition judge the outputs blind for identification and for fidelity. The benchmark is preregistered, needs only published texts used with permission, and carries a stop condition: if grounded agents are not identified more often than the prompted single model, the hypothesis is falsified and the result is published. Working paper, not peer reviewed; comments welcome.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-05
DOI
https://doi.org/10.5281/zenodo.23979452
Primary Topic
Ethics and Social Impacts of AI
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Can AI reason from inside a tradition? A preregistered benchmark for tradition fidelity in language models

Hector Ibarra
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
article

Can AI reason from inside a tradition? A preregistered benchmark for tradition fidelity in language models

Hector Ibarra
article en

Abstract

Language models are increasingly asked to speak for cultures, religions and other ways of knowing, and recent work suggests they often adopt a tradition's voice while reasoning from a narrower default. A 2026 preprint describes a "persona illusion" on ethical dilemmas rooted in Indigenous contexts; Agarwal and colleagues find that an average American is a closer match to Indian cultural values than any Indian-trained model they tested, and that prompting does not meaningfully improve alignment. Whether this gap can be closed by architecture, and not only by instructions, is untested. We propose the WhatIfWe Open Benchmark, which compares three conditions on problems where traditions are known to reason differently: tradition-grounded agents, whose retrieval is partitioned by tradition and whose claims must cite their sources; a single model prompted to answer as each tradition in turn; and a single model answering for all traditions at once. Practitioners formed in each tradition judge the outputs blind for identification and for fidelity. The benchmark is preregistered, needs only published texts used with permission, and carries a stop condition: if grounded agents are not identified more often than the prompted single model, the hypothesis is falsified and the result is published. Working paper, not peer reviewed; comments welcome.

Zenodo (CERN European Organization for Nuclear Research)
Openalex Percentile: Top 16%
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon