Feasibility of Real Time 3D Object Detection for Accessibility Use Cases

Augmented Reality (AR) holds promise for improving visibility and accessibility of real-world objects. AR systems, however, often rely on 2D overlays or marker-based methods, which face limitations such as camera’s poor performance in low-light. We present a new approach to improve visual accessibility in AR. We developed a Unity-based application for the HoloLens 2, integrating Azure Object Anchors (AOA) for real-time 3D object detection with depth sensors. Using pre-scanned 3D models, the application detects and then projects virtual overlays on objects to increase their visibility while also logging eye gaze and spatial movement to enable evaluation of user interactions.