Stereo-Correct Digital Magnification of the Physical World Using Video-Pass-Through Augmented Reality

Video-pass-through (VPT) augmented reality (AR) enables greater field of view and fidelity compared to current optical-see-through AR technologies. However, the computational complexity of VPT AR has traditionally resulted in high latency, low resolution, and/or visual artifacts. Only recently have devices come on the market that successfully mitigate these drawbacks, such as the Varjo XR-3 and XR-4, the Apple Vision Pro, and the Meta Quest 3. We propose that low-latency VPT AR introduces several interesting 3D User Interface (3DUI) capabilities. In this position paper, we discuss one of these 3DUIs, namely stereo-correct digital magnification of the physical environment. Specifically, this paper presents a novel system for stereo-correct digital magnification of the physical environment using the Varjo XR-3 (but that can be generalized to other VPT AR systems). The digital magnification algorithm we propose enables stereoscopic magnification of physical objects in real time by utilizing the front-facing camera feeds of the VPT AR system, implemented entirely through software. In addition, we outline the challenges and limitations associated with the proposed system, discuss our current progress, and suggest potential future directions for the development of our proposed VPT AR digital magnification system.