Jumping to predictions: Auditory and visual prediction in hurdling

In everyday life, there are situations where sensory input can be temporarily unavailable. To maintain a coherent perceptual experience, the brain then relies on internal predictive models, shaped by prior experience and informed by sensory signals from different modalities such as audition and vision. The present study investigated how our brain actively compensates for brief masking of auditory and visual input at the example of hurdling, a complex full-body movement. Participants underwent functional magnetic resonance imaging (fMRI) while watching hurdling videos in which visual and/or auditory information was briefly masked, requiring the reliance on internal models during a movement prediction task. Between two fMRI sessions, participants completed six weeks of hurdling training to strengthen their own sensorimotor models of hurdling. As expected, prediction accuracy declined when sensory input was masked. Under visual masking, the brain not only relied more on the remaining auditory stream but also engaged frontal, motor, and visual regions despite the absence of visual input, hinting at top-down visual prediction. Auditory masks increased the recruiting of visual regions but showed no clear evidence for an auditory analogue of the top-down effect seen under visual masking. Predictive accuracy improved after hurdling training, and the training-related changes in neural activation overlapped with activation patterns under visual masking in frontal control and visuomotor areas, consistent with more efficient internal models for visual prediction. Due to field of view restrictions, no reliable effects could be detected in the cerebellar cortex. Taken together, we demonstrated that the brain flexibly recruited modality-specific networks depending on the available input and, in the case of missing visual information, predictions.