Audiovisual Contributions to the Occurrence of Roll Vection in Virtual Reality

Vection (i.e., the experience of self-motion in the absence of actual motion), has traditionally been considered a visual phenomenon. However, recent work on yaw vection (i.e., illusory rotations around the vertical axis), suggests that auditory cues may contribute to vection as well; specifically, if the sounds are similar to those heard in the real world and come from a sound source that typically remains in the same spatial location. In the present study, we sought to determine if roll vection (i.e., illusory rotations around the longitudinal axis) is also enhanced by auditory cues. To test this possibility, we had 44 participants experience three different combinations of sensory cues (audiovisual, visual-only and auditory-only), which were presented via virtual reality a total of three times each. We found that participants experienced vection sooner and more convincingly the first two times they were exposed to the audiovisual condition relative to the visual-only and auditory-only conditions. However, there was no difference between the audiovisual and visual-only condition the third time they experienced them. Regardless of the number of times the participants experienced each condition, the audiovisual and visual-only conditions were always characterized by higher convincingness ratings and lower onset latencies than the auditory-only condition. In tandem, these results suggest that audiovisual stimuli can indeed elicit roll vection that starts earlier and is more convincing than unisensory variants of the stimuli (i.e., visual-only and auditory-only). However, the benefit of including auditory cues diminishes over the course of repetitions suggesting that the brain may down-weight auditory cues. Future studies are needed to better characterize the mechanisms underlying this finding and other factors which may impact this effect (e.g., sound type and field-of-view size).