Exploring Visual Scanning in Augmented Reality: Perspectives From Deaf and Hard of Hearing Users

Sensory-intensive and attention-demanding tasks like visual scanning, interacting with 3D objects, comprehending and following instructions, etc. are becoming more common in Augmented Reality (AR) environments as the technology expands through diverse fields. It is important to understand how these types of tasks are experienced by Deaf and Hard of Hearing (DHH) people, especially if those tasks involve any sound or compete with attention shifts (e.g., observing someone signing) in both real and virtual environments. Our current research specifically aims to identify the challenges that DHH users encounter when engaging in visual scanning in an AR environment. Using Angry Birds AR as a probe in our research, 11 DHH participants, with varying hearing abilities played seven rounds of the game, followed by a short structured interview and a long semi-structured interview. Our findings revealed that subtle audio cues and excessive visual indicators impacted participants’ performances negatively. Additionally, they positioned themselves strategically for maximum spatial awareness but faced challenges with AR visual cues due to the lighting conditions in the real environment. We further suggested design implications such as customizable, user-friendly haptic and textual feedback, and intelligent spatially aware mechanisms for AR.