AR-in-VR simulator: A toolbox for rapid augmented reality simulation and user research

While providing exciting opportunities for spatial computing, current generation augmented reality (AR) headsets face visual quality and visual comfort issues. Designing headsets that are both perceptually accurate and comfortable to use for extended periods is difficult as these devices rely on state-of-the-art developments in optics, display technology, spatial computing and graphics. Understanding the requirements and parameters needed to create great AR devices currently requires expensive and time consuming prototype development and user research. Here, we present the AR-in-VR Simulator, a tool to facilitate rapid AR device prototyping and user research of the perceptual requirements needed for AR systems by simulating various display, optics and rendering properties of AR systems and presenting them in a virtual reality (VR) headset. Our simulator is suitable for conducting perceptual research on various potential AR device properties and artefacts, such as FOV size and shape, stereoscopic display configurations, and luminance and color non-uniformity. Simulations can be presented in realistic 3D environments, passthrough VR, or even Gaussian Splat environments allowing for a range of naturalistic data collection possibilities. We further demonstrate the utility of the simulator by using it to extend prior work on the perception of partially overlapping stereoscopic AR displays to a 6DOF VR simulation, capturing more ecologically valid and natural user interaction compared to traditional stereoscope based perceptual studies.