Efficient Sign Language Recognition with Skeleton Data: A Study of Keypoint Selection, Pose Estimators, and GCN Models

Sign language is the primary form of communication for deaf individuals; however, not everyone understands it. Sign language is a complex and dynamic communication system that utilizes hand gestures, body postures, and facial expressions to convey information. Therefore, a Sign Language Recognition (SLR) system can help break down communication barriers. In this work, we focus on effectively recognizing sign language using human skeleton data, where each component of the SLR system is carefully examined by studying and evaluating various methods. Firstly, we investigate the impact of different pose estimators on recognition performance. Secondly, a keypoint selection strategy is proposed to extract informative keypoints specifically designed for sign language recognition tasks. Moreover, we assess the role of Graph Convolutional Network (GCN) architectures for this task. In this work, we investigated two advanced GCN architectures, ST-GCN++ and CTR-GCN, to train the model using the selected keypoints. Additionally, label smoothing based on text information is exploited to improve the training process. Our proposal achieved promising results on the WLASL benchmark, achieving an accuracy of 56.46% on WLASL2000, surpassing recent studies that rely solely on skeleton input.