Multimodal Reading Strategies in Native and Advanced L2 Korean Speakers: An Eye-Tracking Study of Text, Image, and Auditory Integration

Objectives: This study investigates the impact of auditory input, specifically within the Reading While Listening (RWL) framework, on the comprehension of multimodal texts among native Korean speakers (L1) and advanced Korean learners (L2). Additionally, it explores differential reading strategies and the influence of working memory (WM) capacity on the processing of multimodal content.Methods: Eye-tracking technology was utilized to examine the eye movement patterns of 28 participants (14 L1, 14 L2) as they engaged with multimodal texts consisting of written, visual, and auditory components. Key metrics— including dwell time, fixation count, and saccadic movements—were analyzed to gain insights into the cognitive processing behaviors of both groups. Furthermore, WM capacity was assessed to determine its effect on the adoption of specific reading strategies. Results: The findings revealed that both L1 and L2 learners predominantly focused on textual rather than visual information, with L2 participants exhibiting significantly higher fixation counts and longer fixation durations, indicative of elevated cognitive load. In terms of strategy use, L2 learners relied more on localized, compensatory strategies (e.g., keywordtriggered switching), whereas L1 participants more frequently employed integrative strategies such as content integration reading. Conclusion: The results suggest that while auditory input in RWL contexts can facilitate multimodal comprehension, it does not fully alleviate the heightened cognitive demands observed among L2 learners. Instructional approaches aiming to address these challenges may benefit from incorporating explicit strategy training and considering learners’ WM capacity to optimize multimodal reading outcomes.