Consideration of artificial intelligence applications for interpreting communicative movements by individuals with visual and/or motor disabilities

Access to speech-language therapies that promote optimal communication outcomes is a fundamental right outlined by the United Nation's Article 19 of the Convention on the Rights of Persons with Disabilities as well as the Communication Bill of Rights of the National Joint Committee on the Communication Needs of Persons with Severe Disabilities. For many individuals, augmentative and alternative communication (AAC) offers an important set of supports for realizing this critical human right. Yet despite a strong evidence base, many AAC systems are not tailored to the support needs of individuals with visual and/or motor impairments. In this paper we explore the potential for artificial intelligence to interpret body-based communicative movements (including gestures) into speech output. The team, which includes individuals with motor impairments, developed and tested a prototype application that shifts the burden of access off the AAC user and onto the application, through the use of body-based sensors. The effort demonstrated proof of concept that the prototype could learn meaningful idiosyncratic gestures and ignore unintentional ones. Movement sensors interpreted the gestures regardless of the user's or communication device's position, reducing constraints on its usability, potentially increasing the flexibility of AAC access. Dedicated research is needed to confirm this possibility.