Using Eye Tracking to Explore How Auditory-Visual Chunking Modulates Visual Search Strategies in Expert Musicians in the Presence of Auditory Interference

Experts show performance advantages during visual search due to their extensive experience with domain-specific stimuli. Experts form memory representations for meaningful visual patterns, called chunks, that group together multiple domain-specific features into larger patterns. The ability of experts to form chunks could allow them to more precisely encode a search template, which could facilitate visual search performance. In the domain of music reading, expert musicians might form chunks that are multimodal (i.e., using visual and auditory modalities). The current studies addressed the possibility that chunks are multimodal by extending a previous cross-modal visual search task to manipulate the presence of auditory interference either during encoding or retrieval, comparing expert and non-musicians’ eye movements. In addition, two exploratory studies compared fixation locations of experts and non-musicians during encoding, as well as correlations between self-report music awareness and memory and eye movement measures. Results showed that, compared to non-musicians, experts had higher accuracy, and this was magnified in the presence of interference during encoding, indicating experts' performance advantages. In addition, compared to non-musicians, experts were able to modulate their search performance in the presence of auditory interference during retrieval only. This was evidenced by faster first fixations on the target to the trial end. Finally, compared to non-musicians, experts show greater effects of relevancy, as indicated by their dwell-based pattern of results across Studies 1 and 2. Fixation patterns also reveal that experts fixate on more relevant regions of a bar of music, in order to encode it into a meaningful pattern or chunk. Not surprisingly, experts, compared to non-musicians had significant correlations between eye movement patterns and self-report measures on musical awareness and memory, suggesting that experts who rate themselves at having better musical memory and awareness of musical structure perform better on a music-related visual search task. Together, these findings suggest that experts are using both auditory and visual information to precisely encode a bar of music into a chunk, allowing them to efficiently search for the target within a search array. The presence of auditory interference impacts non-musicians more, compared to experts, suggesting that expertise modulates auditory distraction.