Exploring the Potential of Multimodal Large Language Models as Augmentative and Alternative Communication Systems: Optimization, Challenges, and Ethical Considerations

Augmentative and Alternative Communication (AAC) supports individuals with speech or language impairments, yet existing solutions often lack real-time adaptability. This paper explores applying multimodal large language models (MLLMs) to enhance AAC systems. Through controlled experiments using synthetic images and varied prompt strategies, we evaluate GPT-4's ability to produce concise, semantically accurate, and fluent expressions for typical AAC needs such as requesting basic items or expressing emotions. Our findings indicate that context-rich or emotion-focused prompts yield more nuanced outputs, although these responses may be lengthier. We also address key ethical and technical considerations, including user privacy, model bias, and hardware constraints. The paper highlights the necessity of user-centered design, robust data protection, and interdisciplinary collaboration to develop further MLLM-driven AAC solutions for individuals with complex communication needs.