Indonesian Sign Language (BISINDO) Recognition Using Spatially Aware Body Gesture Recognition

The population of deaf and hard of hearing (DHH) in Indonesia was approximately 6.9 million people in 2015. Sign language is not widely known or studied by the general public, which segregated the DHH communities from society. The purpose of this paper is to research the feasibility of creating a lightweight Sign Language Recognition system using Deep Learning algorithms specifically for the BISINDO sign language. In this paper we propose using a CNN + GCN + BiLSTM model to be able to learn both the spatial and temporal features of a sign. The model’s performance is evaluated through its accuracy based on the true positive and true negatives prediction over all the predictions. The research resulted in a model with the best accuracy of 90% on the word level. However, several problems are found such as lack of video datasets and within the graph construction method. Based on the experiment results, we suggest that sentence level video datasets are increased to capture true temporal features and for the graph construction method for the GCN to be improved.