Signer-Independent, Real-Time Recognition of the Arabic Sign Language Alphabet from Hand Landmarks in Video

Arabic Sign Language (ArSL) is the main means of communication for many deaf people in the Arab world, and automatic alphabet recognition can support their access to technology and services. However, most existing studies classify static images and evaluate on signers already seen during training, which overestimates real-world performance. To address this, we propose a video-based recognition system that uses hand landmarks only, is evaluated on unseen signers, and is deployed as a live browser–server application. The method uses the 39 alphabet signs of the KArSL-502 dataset, MediaPipe hand landmarks normalized to wrist and palm size, and three models (Transformer, BiLSTM and temporal convolutional network), evaluated with leave-one-signer-out validation, five seeds and exact McNemar tests with Holm correction. Across 15 leave-one-signer-out runs on the three KArSL signers, the augmented Transformer with a 28-frame input achieved a mean accuracy of 79.42 ± 6.81% and a mean top-5 accuracy of 95.34 ± 3.20%. Averaging the five seed predictions within each held-out fold yielded a pooled accuracy of 81.67%. In a single-participant live feasibility test using a separately trained deployment checkpoint, 235 of 255 displayed predictions were correct (92.16%). The median time from the first frame with a detected hand to the displayed result was 1.43 s. These results describe recognition of isolated alphabet signs in the three-signer benchmark and the feasibility of the live prototype under the tested conditions.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-12
DOI
https://doi.org/10.20944/preprints202611.1312.v1
Primary Topic
Hand Gesture Recognition Systems
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Signer-Independent, Real-Time Recognition of the Arabic Sign Language Alphabet from Hand Landmarks in Video

Ibrahim Amer Ibrahim, Sarah Sabah Al-Jubouri
Preprints.org
Hand Gesture Recognition Systems
preprint

Signer-Independent, Real-Time Recognition of the Arabic Sign Language Alphabet from Hand Landmarks in Video

Ibrahim Amer Ibrahim, Sarah Sabah Al-Jubouri
preprint en

Abstract

Arabic Sign Language (ArSL) is the main means of communication for many deaf people in the Arab world, and automatic alphabet recognition can support their access to technology and services. However, most existing studies classify static images and evaluate on signers already seen during training, which overestimates real-world performance. To address this, we propose a video-based recognition system that uses hand landmarks only, is evaluated on unseen signers, and is deployed as a live browser–server application. The method uses the 39 alphabet signs of the KArSL-502 dataset, MediaPipe hand landmarks normalized to wrist and palm size, and three models (Transformer, BiLSTM and temporal convolutional network), evaluated with leave-one-signer-out validation, five seeds and exact McNemar tests with Holm correction. Across 15 leave-one-signer-out runs on the three KArSL signers, the augmented Transformer with a 28-frame input achieved a mean accuracy of 79.42 ± 6.81% and a mean top-5 accuracy of 95.34 ± 3.20%. Averaging the five seed predictions within each held-out fold yielded a pooled accuracy of 81.67%. In a single-participant live feasibility test using a separately trained deployment checkpoint, 235 of 255 displayed predictions were correct (92.16%). The median time from the first frame with a detected hand to the displayed result was 1.43 s. These results describe recognition of isolated alphabet signs in the three-signer benchmark and the feasibility of the live prototype under the tested conditions.

Preprints.org
Hand Gesture Recognition Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon