Real-Time Gesture Recognition to Aid Communication in Children with Motor Impairments

Recognition of hand gestures has become critical in man-machine interaction especially for persons with motor disability who need to make better use of other ways of communication. This paper introduces a strong vision-based hand gesture recognition system with CNN architectures; VGG16,ResNet implemented through the Mediapipe framework to improve the system recognition rates as well as optimize mobile device performance. That is why our system fills the lack of knowing dynamic gestures and high dependence on hardware in the current CNN models by using transfer learning, data augmentation, and TensorFlow Lite. These changes decrease training time and increase generalization as well as decrease the cost, thereby making the model acceptable and efficient for the teaching environment. Findings from experiment suggest that the proposed model has high recall and precision as participants' validation data indicates an accuracy of more than 98% the proposed gesture based communication model may benefit children with motor disabilities by providing them with a flexible and accurate form of communication.