Influence of simple crossmodal correspondence on interpretation of spoken intent

Crossmodal correspondences (CMCs) are consistent associations between sensory features from different modalities. Previous research has demonstrated that these pairings can be recruited in the resolution of the Rubin face-vase illusion. This research builds upon and expands these findings into the resolution of language ambiguity. The study uses the feature pairings of elevation/pitch, lightness/pitch, and size/pitch to influence the perceived intonation of a spoken utterance. Participants listened to a range of auditory stimuli, varying in their intent, and attempted to classify them as questions or statements. We found that change in the irrelevant visual features of elevation or lightness could bias their judgements in a manner consistent with an actual change in the pitch of the auditory stimulus. Size/pitch does not appear to affect language judgements, and we confirm this in a series of follow-up experiments. This suggests that simple crossmodal correspondences can influence high-level ambiguity resolution, at least in some cases. These findings have important implications for the use of visual cues in audiovisual language studies such as the visual world paradigm.