Sign Language Recognition Using Enhanced Feature Extraction and Classification Using a Hybrid Architecture
Sign language conveys meaning through both manual gestures (hand shapes and movements) and non-manual cues such as facial expressions and body posture, yet previous studies have often modeled these channels separately, which can lose meaning. This study proposes a hybrid deep learning architecture that recognizes manual and non-manual gestures jointly. Each video is divided into 16-frame segments; YOLOv8 localizes the face, VGG16 and a recurrent neural network extract spatial and temporal facial features, and an improved 3D convolutional network (C3D) captures full-body spatiotemporal motion. The fused features are classified by an improved bidirectional long short-term memory network (iBiLSTM). The framework was evaluated on three benchmark datasets, PkSLMNM, RWTH-PHOENIX-Weather 2014T, and SILFA, using 5-fold cross-validation. On PkSLMNM, the model achieved a mean cross-validation accuracy of 95.79% (standard deviation 0.37%), outperforming LSTM (86.12%) and BiLSTM (89.15%) baselines, and it reached 93.84% and 94.21% accuracy on RWTH-PHOENIX-Weather 2014T and SILFA, respectively. An ablation study showed that fusing facial and body features outperformed either modality alone. These results indicate that combining manual and non-manual cues in a unified architecture improves both the accuracy and the cross-dataset robustness of dynamic sign language recognition.
Authors
- Sameena Javaid (ORCID: https://orcid.org/0000-0001-8017-7196)
- Muhammad Talha Ubaid (ORCID: https://orcid.org/0000-0003-3632-5176)
- Mansoor Alam (ORCID: https://orcid.org/0000-0001-5773-7140)
- Mazliham Mohd Suud
- Safdar Rizvi
Publication Details
- Journal
- Preprints.org
- Published
- 2026-11-18
- DOI
- https://doi.org/10.20944/preprints202611.1796.v1
- Primary Topic
- Hand Gesture Recognition Systems
- Type
- preprint