fingARspell: A Mobile AR Tool for Learning the Deafblind Manual Alphabet

Deafblind manual, also known as tactile fingerspelling, is one of the many communication methods used to communicate with individuals living with the combination of sight and hearing impairments (i.e. deafblindness). Access to comprehensive resources and instructors in Deafblind manual can be limited and mostly rely on traditional learning methods such as textbooks. Therefore, we propose fingARspell, a mobile AR tool for learning the tactile fingerspelling alphabet through camera-based hand-tracking and a virtual representation of different tactile signs for each letter of the alphabet using augmented reality. fingARspell aims to provide a more immersive and visually engaging experience for learning the Deafblind manual alphabet with the use of mobile augmented reality technology. This work presents the design and implementation of the system, as well as the results of a preliminary evaluation with 20 participants, including the feedback of 5 expert teachers of Deafblind manual.