Neural effects of expectation violation generalise across sensory modalities

The brain receives more sensory information than it can usefully employ to control behaviour. This sensory overload can be reduced by exploiting regularities in the environment to predict future events. Previous work on the role of prediction in perception has mostly focused on events within a single sensory modality. Here, we asked whether expectation violations in one sensory modality (audition) can affect neural representations of stimuli in a separate modality (vision). Human observers viewed rapid sequences of randomly-oriented visual gratings together with a task-irrelevant stream of alternating high- and low-pitched tones. Using electroencephalography (EEG) to measure functional brain activity, we verified a robust neural mismatch response to unexpected sounds, consistent with previous studies. Critically, using machine learning to extract neural orientation tuning to gratings, we observed stronger tuning following unexpected sounds, suggesting that unimodal mismatches trigger cross-modal updating. The findings suggest that predictive coding theories of perception should incorporate cross-modal influences on internal models of the sensory environment.