Sign Language Recognition Using Enhanced Feature Extraction and Classification Using a Hybrid Architecture

Sign language conveys meaning through both manual gestures (hand shapes and movements) and non-manual cues such as facial expressions and body posture, yet previous studies have often modeled these channels separately, which can lose meaning. This study proposes a hybrid deep learning architecture that recognizes manual and non-manual gestures jointly. Each video is divided into 16-frame segments; YOLOv8 localizes the face, VGG16 and a recurrent neural network extract spatial and temporal facial features, and an improved 3D convolutional network (C3D) captures full-body spatiotemporal motion. The fused features are classified by an improved bidirectional long short-term memory network (iBiLSTM). The framework was evaluated on three benchmark datasets, PkSLMNM, RWTH-PHOENIX-Weather 2014T, and SILFA, using 5-fold cross-validation. On PkSLMNM, the model achieved a mean cross-validation accuracy of 95.79% (standard deviation 0.37%), outperforming LSTM (86.12%) and BiLSTM (89.15%) baselines, and it reached 93.84% and 94.21% accuracy on RWTH-PHOENIX-Weather 2014T and SILFA, respectively. An ablation study showed that fusing facial and body features outperformed either modality alone. These results indicate that combining manual and non-manual cues in a unified architecture improves both the accuracy and the cross-dataset robustness of dynamic sign language recognition.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-18
DOI
https://doi.org/10.20944/preprints202611.1796.v1
Primary Topic
Hand Gesture Recognition Systems
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Sign Language Recognition Using Enhanced Feature Extraction and Classification Using a Hybrid Architecture

Sameena Javaid, Muhammad Talha Ubaid, Mansoor Alam, Mazliham Mohd Suud et al.
Preprints.org
Hand Gesture Recognition Systems
preprint

Sign Language Recognition Using Enhanced Feature Extraction and Classification Using a Hybrid Architecture

Sameena Javaid, Muhammad Talha Ubaid, Mansoor Alam, Mazliham Mohd Suud, Safdar Rizvi
preprint en

Abstract

Sign language conveys meaning through both manual gestures (hand shapes and movements) and non-manual cues such as facial expressions and body posture, yet previous studies have often modeled these channels separately, which can lose meaning. This study proposes a hybrid deep learning architecture that recognizes manual and non-manual gestures jointly. Each video is divided into 16-frame segments; YOLOv8 localizes the face, VGG16 and a recurrent neural network extract spatial and temporal facial features, and an improved 3D convolutional network (C3D) captures full-body spatiotemporal motion. The fused features are classified by an improved bidirectional long short-term memory network (iBiLSTM). The framework was evaluated on three benchmark datasets, PkSLMNM, RWTH-PHOENIX-Weather 2014T, and SILFA, using 5-fold cross-validation. On PkSLMNM, the model achieved a mean cross-validation accuracy of 95.79% (standard deviation 0.37%), outperforming LSTM (86.12%) and BiLSTM (89.15%) baselines, and it reached 93.84% and 94.21% accuracy on RWTH-PHOENIX-Weather 2014T and SILFA, respectively. An ablation study showed that fusing facial and body features outperformed either modality alone. These results indicate that combining manual and non-manual cues in a unified architecture improves both the accuracy and the cross-dataset robustness of dynamic sign language recognition.

Preprints.org
Hand Gesture Recognition Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon