Breaking Barriers: Enhancing Learning Outcomes for Deaf and Hard of Hearing Students through Augmented Reality Captioning

In the educational landscape, Deaf and Hard of Hearing (DHH) students often encounter substantial challenges in demonstrating active learning behaviors, thereby affecting learning performance. This research investigated the potential of augmented reality (AR) smart glasses equipped with captioning capabilities in mitigating these difficulties by overlaying real-time, speech-to-text captions onto students’ field of view. In our study, AR provided dense and continuous information, which required further processing, leading to unclear effectiveness. We adopted a classroom-ready AR captioning system and assessed its effectiveness through a randomized field experiment at a school for the deaf. The results indicated that AR captioning could enhance learning performance by removing visual constraints and promoting active learning behaviors. However, this benefit was only observed in contexts where AR captioning provided nuanced information and for DHH students with strong visual and auditory processing abilities. The mechanisms were further demonstrated through surveys and video data.