Audiovisual modulation in multi-agent motion dynamics during virtual world co-navigation in immersive rooms

Temporal modulation of auditory and visual motion perception has been extensively studied using personalized virtual reality systems, i.e., head-mounted devices. However, the experiential conditions of headsets limit the ecological validity of such modulation to an egocentric reference frame, and its variability under the physical co-presence of other individuals is often overlooked. In this work, we investigate the influence of moving audiovisual stimuli on human agent movement in immersive rooms during co-navigation of virtual worlds. We situate our work in three game-based environments resembling real-world scenarios, each populated with virtual soundscapes and salient visual and bimodal objects, the so-called spatial beacons. In our approach, we first simulate the agents’ head and body movements in the immersive room as a variant of the drift-diffusion model (DDM). Using an integrated digital twin, we then capture motion-tracking data for human participants as they freely explore the spatially augmented environments in situ. By comparing simulated and real data, we observe the modality-specific influence of the moving beacons in modulating the emergent motion patterns of the human agents, from which we yield insights about how human interaction in the immersive rooms is shaped by the spatial perception of their augmented surroundings. [Work supported by NSF IIS-1909229 & CNS-1229391.]