The Noisy Encoding of Disparity (NED) Model Predicts Perception of the McGurk Effect in Native Japanese Speakers

The McGurk effect is widely used to demonstrate the importance of visual information in speech perception. Observers view a face enunciating a syllable while hearing an auditory recording of an incongruent syllable, evoking a fusion percept different than either component (McGurk & MacDonald, 1976). The Noisy Encoding of Disparity (NED) model uses probabilistic inference to predict perception by assuming that different McGurk stimuli have fixed stimulus disparities which observers encode with noise and compare to an internal threshold (Magnotti & Beauchamp, 2015). The NED model has been shown to accurately predict McGurk perception in diverse populations, including U.S. college students, children in the United Kingdom (Hirst et al., 2018), and cochlear implant users in Germany (Stropahl et al., 2017). To examine whether the NED model could predict McGurk perception in a non-Western population, 82 native Japanese-speaking participants were recruited from two Japanese universities. Using online testing, 15 different McGurk stimuli were presented 10 times each, along with 30 audiovisual congruent stimuli, in random order. Participants provided typed responses and no feedback was ever given. For control congruent stimuli, accuracy was uniformly high (96%). In contrast, the McGurk effect was highly variable across stimuli and participants. The weakest stimulus evoked the fusion percept on 3% of trials compared with 77% for the strongest stimulus. For more than half of participants, the strongest McGurk stimulus induced the illusion on every trial. Across participants, the NED model accurately predicted the rate of fusion responses for held-out stimuli, mean r^2 = 0.71 ± 0.09, p < 10^-17. The effects of cultural and linguistic factors in shaping the McGurk effect remain a subject of debate. These results demonstrate similarly high levels of variability in the McGurk effect for Western and non-Western populations and show that the NED model can accurately predict perception in both populations.