Dynamic Kannada Sign Language Recognition on Resource Constrained Devices

Sign Language Recognition (SLR) systems have gained significant popularity in recent years. Despite the existence of various Deep Learning (DL) models to classify sign languages, the deployment of these models on resource-constrained devices remains a challenge. Majority of the research on SLR focuses on popular sign languages, such as American Sign Language (ASL), German Sign Language (GSL) etc., while Kannada Sign Language (KSL) remains significantly underexplored. This study focuses on creating a word-level custom dataset of 33 classes consisting of 2319 videos and training three DL models, namely, LSTM, BiLSTM, and Encoder-only Transformer for dynamic KSL recognition. Trained models were then optimized with quantization techniques and converted to .tflite format, for efficient deployment on resource-constrained devices like smartphones. Dynamic Post Training Quantization (PTQ) model, for the BiLSTM model achieved the highest accuracy of 95.71%, followed by the LSTM model with 94.7%, and the Transformer-based architecture achieved 94.19%. Transformer model deployed on smartphone achieved the smallest inference time of 16.2 ms with a model size of 1097.77 KB, followed by LSTM recording 19.6 ms as inference time with a model size of 1254.49 KB and BiLSTM recording the highest inference time of 27.8 ms with a model size of 3862.01 KB.