Performance metrics of real-world perception in augmented reality

The emerging technology of augmented reality (AR) headsets allows users to interact with virtual content that is stereoscopically superimposed on a view of their surroundings. This is achieved either with transparent displays that allow a direct view of the world ("see-through"), or with cameras that stream video to conventional displays inside the headset ("pass-through"). We sought to benchmark perception of the real world through various commercial AR devices, as well as quantify any discomfort that may result from their prolonged use. To this end, we developed a suite of perception and action tasks that provide performance metrics for complementary aspects of the visual experience. Tasks included color matching, hand-eye coordination, contrast sensitivity and acuity measurements, motion discrimination, visual-field measurements, catching balls, and a demanding locomotion task. The selection and design of these tasks was informed by the anticipated pros and cons of see-through and pass-through systems, such as latency of pass-through devices, and the reduced field of view in see-through headsets. Participants (N=24) performed the tasks while wearing different headsets (Magic Leap 2, Meta Quest Pro/3, Varjo XR3) and in a naked-eye control condition. A single session comprising all tasks took about one hour. To prevent carry-over of potential discomfort, sessions were conducted on separate days. We assessed visual discomfort using a standardized questionnaire that was administered before and after each session, as well as interview style questions. Our results establish the benchmark suite as sensitive and comprehensive, and provide detailed quantifications of the pros and cons of each technological approach in relation to the naked eye baseline.