Effects of emotion and emotional congruence on the multisensory temporal binding window during audiovisual speech integration

The integration of auditory and visual cues supports efficient speech and emotion perception, yet little is known about how emotional salience interacts with temporal congruency during audiovisual binding. In this study, 125 participants completed a synchrony judgment task using dynamic face–voice pairs varying in emotion (angry, happy, neutral), congruency, and temporal alignment. Temporal binding windows (TBWs), or the temporal offsets within which a stimulus pair is perceived as perceptually bound, were calculated for each condition. TBWs were significantly wider for emotionally salient (angry, happy) compared to neutral congruent pairs, indicating greater tolerance for temporal asynchrony in the presence of emotion. Moreover, emotionally congruent pairs yielded wider TBWs than incongruent pairs, demonstrating that emotional congruence influences low-level perceptual aspects of multisensory integration. These results suggest that emotion, as a higher-order cue to bind, interacts with low-level temporal information to bias causal inference toward integration. Our findings extend models of multisensory integration by highlighting the interaction of emotion and low-level stimulus properties as a dimension shaping audiovisual binding. This work has implications for understanding social communication and may provide insight into populations with known difficulties in both emotion perception and multisensory integration.