Visual attention in bilingual instructional videos: effects of audiovisual congruency and subtitle language

How do subtitles shape where learners look, and what they learn, in bilingual instructional video? Subtitles are increasingly central to instructional video in multilingual education, where learners must integrate spoken input with written text. In the present paper, we recorded eye movements while university students watched two-speaker instructional videos to test the effects of audio language (original L2 vs L1 dubbed) and subtitle language (L1 vs L2). Experiment 1a (N = 40 Spanish L1, English L2) contrasted original English audio with Spanish-dubbed versions, and we found that dubbing shifted gaze from the speakers' mouths to their eyes; comprehension was also higher for dubbed L1 than original L2. Experiment 1b tested the same Spanish-speaking participants with English-audio videos containing either English (L2) or Spanish (L1) subtitles, and included native English speakers viewing the English-audio/English-subtitle condition as a comparison group. In both groups, participants consistently devoted most of their gaze to subtitles rather than to the speakers' eyes or mouths, independent of subtitle language; comprehension was equally high for L1 and L2 subtitles. Taken together, these findings indicate the key role of subtitles in guiding visual attention, with direct implications for the design of bilingual multimedia instruction.