A Multimodal Machine Learning-Based Assistive Communication App for Impaired Children Using TensorFlow Lite

This study presents the development and evaluation of a mobile-based multimodal assistive communication application designed to enhance communication for impaired children. The application integrates gesture recognition, facial expression analysis, customizable communication buttons, language video tutorials, and text-to-speech functionality to facilitate real-time, interactive communication. The research employed a user-centered design methodology, including online surveys, focus group discussions, and iterative prototype testing with 15 participants, comprising children, caregivers, and professionals. A comprehensive Software Requirements Specification (SRS) guided the development, ensuring alignment with user needs and practical usability in both home and educational settings. System evaluation was conducted following the ISO/IEC 25010:2023 software quality framework, emphasizing six attributes: functional suitability, performance efficiency, interaction capability, usability, compatibility, and portability. Findings revealed strong agreement across all evaluated criteria, with an overall weighted mean of 4.57. The application demonstrated high adaptability to user preferences, consistent performance across devices, reliable real-time interaction, and an intuitive interface suitable for users with varying digital literacy. Caregivers and professionals reported that the system effectively supports communication in dynamic, real-world scenarios, reduces frustration, and strengthens social and educational engagement for impaired children. Based on these outcomes, it is recommended to further enhance responsiveness and contextual accuracy, expand multilingual and culturally relevant features, and improve integration with third-party assistive technologies. Structured training programs for caregivers and longitudinal evaluations are also advised to optimize adoption and measure long-term impact. Overall, the system offers a scalable, accessible, and inclusive tool that addresses communication barriers, empowers non-verbal users, and provides a reliable platform for educational and therapeutic support.