AI-driven Augmentative and Alternative Communication System for Communication Enhancement in Cerebral Palsy Using American Sign Language Recognition

Augmentative and alternative communication (AAC) methods are designed to facilitate effective communication for individuals with speech, language, or writing impairments.We introduce a novel AAC system designed to enhance communication for individuals with speech, language, or writing impairments, such as those with cerebral palsy (CP).Addressing the limitations of existing AI-driven AAC solutions, advanced AI and machine learning techniques are integrated in the system to minimize user effort in conveying their thoughts.Utilizing a readily available webcam as a key input modality, the system employs MediaPipe to capture hand gestures corresponding to American Sign Language signs.The resulting visual data is then processed and classified by a random forest model.By interpreting these sensor-captured gestures, the system enables users to input partial vocabulary, which subsequently prompts generative AI models to predict and complete intended text.Empirical evaluations, conducted through two distinct experiments, validate the system's viability and demonstrate its potential to significantly improve communication accessibility for individuals with CP through an accessible and intuitive gesture-based interface.