Annotator Disagreement as a Calibrated Language-Identity Signal for Code-Switched Speech Recognition in Kreol Morisien

In code-switched speech between lexically related languages, the language of a word can be graded rather than categorical. We propose that disagreement between human annotators marks these positions and can serve as a calibration target. On a trilingual Kreol Morisien, French, and English broadcast corpus, a classifier trained to reproduce two annotators’ label distributions is well calibrated (expected calibration error 0.008) and, under cross-validation, is more uncertain at the 100 words they disputed than at agreed words (entropy ratio 1.68). We use this signal to gate language-specific low-rank adapters in the decoder of a fine-tuned Whisper large-v3 model, with controls that separate the effects of decoding, adaptation, and the signal itself. Standard decoding inserts an automatically detected language token the fine-tuned model was never trained with; decoding without it accounts for essentially the whole of the overall improvement over the baseline. Against a control decoded identically, language conditioning redistributes errors between the languages rather than reducing them: French word error rate (WER) falls by 1.5 to 2.0 points across three training seeds, Kreol Morisien WER rises by about 0.3 points, overall WER changes by less than 0.09 points, and switch-point error does not improve. Ablations locate the French effect in the decode-time signal: replacing it with a uniform, language-swapped, or shuffled signal returns the system to the level of the uniform control, a model trained without language information, and most of the French gain persists when the classifier’s input transcription is degraded to 56% WER. The annotators’ own labels do no better than the predicted signal, and a one-hot form performs comparably, so the recognition effect does not require the signal’s graded form. An error analysis traces the direction of the redistribution to the baseline’s dominant error class: French words written in Kreol Morisien spelling, which are five times more frequent than the reverse. An equivalence lexicon judged by two native speakers attributes 0.6 to 1.0 points of the baseline’s 11.58% WER to spelling convention rather than recognition. These results rest on one dependence that the design cannot remove: the evaluation annotations also train the classifier whose pseudo-labels train the adapters. The uniform control bounds what this dependence contributes. To our knowledge, this is the first use of annotator disagreement as a calibration target for language identification, and the first language-identity conditioning of a speech recogniser for a creole and its lexifier.