A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language

Many isolated words in Chinese Sign Language (CSL) exhibit significant feature similarities, which are primarily conveyed through hand, face, and body movements. Among these, hand features are particularly crucial, as they carry substantial information about the sign language words. However, the spatial and temporal dimensions involved in sign language expressions are often relatively small, leading to a similarity problem that increases the difficulty of sign language recognition. The general extraction and recognition of sign language features can result in confusion when distinguishing similar words, and fine-grained feature extraction of sign language actions may incur high computational costs. To deal with these challenges, this paper proposes a dual-stream deep learning model built on Spatio-Temporal Graph Convolutional Network-Long Short-Term Memory(STGCN-LSTM), which aims to capture both the local features of sign language and the global spatio-temporal characteristics of sign words. The model focuses on capturing both local features in sign language and the overall spatio-temporal characteristics of sign language words. By learning four key sign language phonetic features—hand shape, hand position, hand orientation, and hand motion trajectory—as well as spatio-temporal features derived from whole-body skeletal data, the model aims to improve the recognition of Chinese Sign Language. In this paper, the effectiveness of the proposed model is validated on the SRL500 dataset and the sub-dataset of similar SRL500 consisting of similar words selected from it, and the recognition accuracies obtained are 95.2% and 93.0%, respectively.