Improving Indonesian sign language recognition using lightweight deep learning architectures with knowledge distillation method

Bahasa Isyarat Indonesia (BISINDO) is a sign language that aids in communicating with Indonesia’s deaf community by providing visual communication using hand movement and gestures. Unfortunately, advancements in this field remain stagnant and the quality of life of the Deaf and Hard-Hearing community in Indonesia lack progress, especially in the development of efficient computer models for sign language recognition to facilitate communication. In this paper, we aim to develop and compare several lightweight deep learning-based sign language recognition models using knowledge-distillation (KD), a model compression technique that distills knowledge from bigger deep learning models in various forms to a much smaller and more efficient lightweight models. The result from our experiments have shown EfficientNetB5’s superior performance in terms of pure effectiveness with the highest overall accuracy of 96.94% with an F1-Score of 0.9693, alongside its effectiveness at distilling knowledge to several lightweight deep learning models. As for the lightweight distilled models, MobileNetV3 displayed its excellent ability at balancing efficiency and effectiveness, where the distilled model trained with our KD loss function achieved an accuracy of 83.45% and F1-Score of 0.8331 with only a model size of 3.8 MB and 0.1186 GFLOPs, whereas ShuffleNetV2 provides the most effective result out of all student models, as the distilled model achieved an accuracy of 89.87% and F1-Score of 0.8987 with a slightly higher model size of 6.4 MB and 0.5391 GLOPs compared to MobileNetV3.