Measuring Interpretive Divergence: IVD, a Model-Agnostic and Reference-Free Metric for Machine Translation of Ancient Texts

{"A":[0,347],"machine":[1],"translation":[2],"of":[3,55,108,133,148,213,219,248,370],"an":[4,149,309,435],"ancient":[5],"text":[6,358,401],"is":[7,65,77,85,99,120,175,238,340,381],"presented":[8],"to":[9,35,116,216],"its":[10,329,332,337],"reader":[11],"as":[12,87,182,256,259,383],"a":[13,30,42,67,88,101,122,158,163,195,260,287,291,325,384,389,417,422,453],"single":[14,31],"fluent":[15,357],"string.":[16],"That":[17],"fluency":[18],"conceals":[19],"interpretive":[20,53,135],"risk.":[21],"Where":[22],"the":[23,56,105,131,144,172,217,240,270,318,363,379,429,443],"source":[24,58,461],"genuinely":[25],"admits":[26],"several":[27],"coherent":[28],"readings,":[29],"output":[32,187],"silently":[33],"commits":[34],"one.":[36],"We":[37,427],"introduce":[38],"Inter-Voice":[39],"Divergence":[40],"(IVD),":[41],"model-agnostic,":[43],"reference-free,":[44],"deterministic":[45],"metric":[46,278],"that":[47,296,355,394],"measures":[48],"how":[49],"much":[50],"N":[51],"independent":[52],"readings":[54],"same":[57],"passage":[59,123,202],"diverge":[60],"from":[61,312,421],"one":[62,204,416],"another.":[63],"IVD":[64,84,395,444],"not":[66,100,167,258,293,360,387],"translation-quality":[68],"score":[69],"and":[70,94,157,336,376,434,459],"makes":[71],"no":[72,81,263],"claim":[73],"about":[74],"which":[75,237],"reading":[76],"correct;":[78],"it":[79],"requires":[80],"ground":[82],"truth.":[83],"reported":[86],"two-metric":[89],"architecture:":[90],"IVD-S":[91],"(semantic,":[92],"BERTScore-based)":[93],"IVD-L":[95,98],"(lexical,":[96],"Jaccard-based).":[97],"new":[102],"algorithm":[103],"but":[104],"lexical":[106],"operator":[107],"Paper":[109],"1,":[110],"our":[111,154],"companion":[112],"methodological":[113],"paper,":[114],"elevated":[115],"co-equal":[117],"reporting.":[118],"This":[119,440],"because":[121],"may":[124],"be":[125],"semantically":[126],"convergent":[127],"yet":[128,294,388],"lexically":[129],"divergent,":[130],"signature":[132],"legitimate":[134],"paraphrase.We":[136],"demonstrate":[137],"this":[138,170,255,266,282],"measurement":[139,283],"layer":[140],"model-agnostically":[141],"by":[142],"scoring":[143,430],"Akkadian":[145],"NMT":[146],"outputs":[147],"independent,":[150],"published":[151],"system":[152,164,458],"alongside":[153],"own":[155],"voice":[156],"human":[159],"scholarly":[160],"translation,":[161],"refereeing":[162],"we":[165,253,285],"did":[166],"build.":[168],"On":[169],"corpus":[171,201],"above-chance":[173],"finding":[174],"asymmetry-detection:":[176],"68/1,185":[177],"divergent":[178,192,250],"passages":[179],"(5.74%)":[180],"classify":[181],"asymmetric-divergence":[183],"(the":[184],"shipped":[185],"tool's":[186],"label,":[188],"lens-asymmetric),":[189],"against":[190,344],"0/454":[191],"triples":[193],"in":[194,265,449],"550-triple":[196],"pre-registered":[197,348],"random-triple":[198],"baseline":[199],"(each":[200],"contributes":[203],"such":[205],"three-voice":[206],"triple;":[207],"z":[208],"=":[209],"5.21,":[210],"one-sided":[211],"p":[212],"approximately":[214],"10":[215],"power":[218],"minus":[220],"7;":[221],"Section":[222,229],"6a).":[223],"(A":[224],"guard-matched":[225],"recompute":[226],"strengthens":[227],"this;":[228],"4.1.)":[230],"The":[231,322],"dominant":[232],"remainder":[233],"diverges":[234],"symmetrically":[235],"(94.26%),":[236],"also":[239],"classifier's":[241],"default":[242],"regime":[243],"on":[244,281,456],"unrelated":[245,356],"material":[246],"(100.0%":[247],"454":[249],"random":[251,351,372],"triples);":[252],"report":[254],"context,":[257],"finding,":[261],"since":[262],"test":[264],"paper":[267,441],"distinguishes":[268,297],"whether":[269],"asymmetric":[271],"detections":[272],"measure":[273],"model":[274,313],"projection":[275,314],"or":[276],"short-text":[277],"noise.":[279],"Building":[280],"layer,":[284],"register":[286],"candidate":[288],"lens-symmetry":[289],"diagnostic:":[290],"signature,":[292],"expert-validated,":[295],"genuine":[298],"textual":[299],"ambiguity":[300],"(source-intrinsic":[301],"ambiguity,":[302],"symmetric-divergence;":[303],"what":[304],"some":[305],"prior":[306,450],"discussion":[307],"calls":[308],"\\"ontological\\"":[310],"reading)":[311],"(system-specific":[315],"deviation,":[316],"asymmetric-divergence;":[317],"corresponding":[319],"\\"epistemic\\"":[320],"reading).":[321],"diagnostic":[323],"carries":[324],"decision":[326],"rule":[327,380],"for":[328,342,447],"low-divergence":[330],"case;":[331],"thresholds":[333],"are":[334,397],"provisional,":[335],"falsification":[338],"criterion":[339],"registered":[341,418],"calibration":[343,385],"expert":[345],"labels.":[346],"null":[349],"over":[350],"sentence":[352],"pairs":[353],"finds":[354],"does":[359],"trivially":[361],"satisfy":[362],"rule's":[364],"positive":[365],"branch":[366],"at":[367],"scale":[368],"(7.69%":[369],"520":[371],"pairs;":[373],"Sections":[374],"3.4":[375],"8),":[377],"so":[378],"offered":[382],"target,":[386],"validated":[390],"test.We":[391],"further":[392],"show":[393],"regimes":[396],"corpus-characteristic:":[398],"different":[399,405],"cuneiform":[400],"classes":[402],"produce":[403],"systematically":[404],"divergence":[406],"signatures,":[407],"illustrated":[408],"with":[409],"four":[410],"manuscript":[411],"case":[412],"studies":[413],"(three":[414],"measured,":[415],"prediction),":[419],"drawn":[420],"first-edition-program":[423],"set":[424],"(Section":[425],"6).":[426],"release":[428],"tool":[431],"under":[432],"AGPL-3.0":[433],"open,":[436],"versioned":[437],"benchmark":[438],"(IVD-Bench).":[439],"generalizes":[442],"instrument":[445],"introduced":[446],"Sumerian":[448],"work":[451],"into":[452],"method":[454],"usable":[455],"any":[457,460],"language.":[462]}

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-15
DOI
https://doi.org/10.5281/zenodo.22777435
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
preprint

Measuring Interpretive Divergence: IVD, a Model-Agnostic and Reference-Free Metric for Machine Translation of Ancient Texts

Bodo A. Moser
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
preprint

Measuring Interpretive Divergence: IVD, a Model-Agnostic and Reference-Free Metric for Machine Translation of Ancient Texts

Bodo A. Moser
preprint en

Abstract

A machine translation of an ancient text is presented to its reader as a single fluent string. That fluency conceals interpretive risk. Where the source genuinely admits several coherent readings, a single output silently commits to one. We introduce Inter-Voice Divergence (IVD), a model-agnostic, reference-free, deterministic metric that measures how much N independent interpretive readings of the same source passage diverge from one another. IVD is not a translation-quality score and makes no claim about which reading is correct; it requires no ground truth. IVD is reported as a two-metric architecture: IVD-S (semantic, BERTScore-based) and IVD-L (lexical, Jaccard-based). IVD-L is not a new algorithm but the lexical operator of Paper 1, our companion methodological paper, elevated to co-equal reporting. This is because a passage may be semantically convergent yet lexically divergent, the signature of legitimate interpretive paraphrase.We demonstrate this measurement layer model-agnostically by scoring the Akkadian NMT outputs of an independent, published system alongside our own voice and a human scholarly translation, refereeing a system we did not build. On this corpus the above-chance finding is asymmetry-detection: 68/1,185 divergent passages (5.74%) classify as asymmetric-divergence (the shipped tool's output label, lens-asymmetric), against 0/454 divergent triples in a 550-triple pre-registered random-triple baseline (each corpus passage contributes one such three-voice triple; z = 5.21, one-sided p of approximately 10 to the power of minus 7; Section 6a). (A guard-matched recompute strengthens this; Section 4.1.) The dominant remainder diverges symmetrically (94.26%), which is also the classifier's default regime on unrelated material (100.0% of 454 divergent random triples); we report this as context, not as a finding, since no test in this paper distinguishes whether the asymmetric detections measure model projection or short-text metric noise. Building on this measurement layer, we register a candidate lens-symmetry diagnostic: a signature, not yet expert-validated, that distinguishes genuine textual ambiguity (source-intrinsic ambiguity, symmetric-divergence; what some prior discussion calls an "ontological" reading) from model projection (system-specific deviation, asymmetric-divergence; the corresponding "epistemic" reading). The diagnostic carries a decision rule for its low-divergence case; its thresholds are provisional, and its falsification criterion is registered for calibration against expert labels. A pre-registered null over random sentence pairs finds that unrelated fluent text does not trivially satisfy the rule's positive branch at scale (7.69% of 520 random pairs; Sections 3.4 and 8), so the rule is offered as a calibration target, not yet a validated test.We further show that IVD regimes are corpus-characteristic: different cuneiform text classes produce systematically different divergence signatures, illustrated with four manuscript case studies (three measured, one a registered prediction), drawn from a first-edition-program set (Section 6). We release the scoring tool under AGPL-3.0 and an open, versioned benchmark (IVD-Bench). This paper generalizes the IVD instrument introduced for Sumerian in prior work into a method usable on any system and any source language.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.