Analysis of Vision Transformers for American Sign Language Recognition in Unconstrained Visual Conditions

Sign language is a spoken language for deaf people, but it becomes a tedious task for us to learn this language, thus creating a communication gap between them and us. The solution for this problem is automatic sign language recognition (SLR) systems that focus on translating hand gestures into their linguistic representations. But it is challenging for these systems to work in real-time due to noise in backgrounds, lighting effects, and having unbalanced datasets. The motivation of this research is to address these challenges with the help of Vision Transformer (ViT) architectures. We focus on static American Sign Language (ASL) benchmark datasets having different levels of complex backgrounds and human-induced noise. In this proposed work, stratified K-fold cross-validation and early stopping are used to reduce overfitting and tackle class imbalance, thus promoting model robustness. From experimental analysis, the ViT-based approach shows strong validation performance but notable generalization gaps when evaluated on inference datasets with a restricted variety or gesture classes that are similar in appearance. Although this is a limitation for deployment in real-time, findings reveal the importance of designing human-noise datasets that are deployment-oriented. This work describes the strengths and limitations of attention-based models and provides insights for designing a more robust framework for assistive communication applications.

Authors

Institutions

Publication Details

Journal
International Journal of Scientific Engineering and Research
Published
2026-11-21
DOI
https://doi.org/10.70729/se261119225208
Primary Topic
Hand Gesture Recognition Systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Analysis of Vision Transformers for American Sign Language Recognition in Unconstrained Visual Conditions

Sumit Budhiraja, Sunil Agrawal, Satwinder Kaur, Garima Joshi
3 links
International Journal of Scientific Engineering and Research
Hand Gesture Recognition Systems
article

Analysis of Vision Transformers for American Sign Language Recognition in Unconstrained Visual Conditions

Sumit Budhiraja, Sunil Agrawal, Satwinder Kaur, Garima Joshi
article en

Abstract

Sign language is a spoken language for deaf people, but it becomes a tedious task for us to learn this language, thus creating a communication gap between them and us. The solution for this problem is automatic sign language recognition (SLR) systems that focus on translating hand gestures into their linguistic representations. But it is challenging for these systems to work in real-time due to noise in backgrounds, lighting effects, and having unbalanced datasets. The motivation of this research is to address these challenges with the help of Vision Transformer (ViT) architectures. We focus on static American Sign Language (ASL) benchmark datasets having different levels of complex backgrounds and human-induced noise. In this proposed work, stratified K-fold cross-validation and early stopping are used to reduce overfitting and tackle class imbalance, thus promoting model robustness. From experimental analysis, the ViT-based approach shows strong validation performance but notable generalization gaps when evaluated on inference datasets with a restricted variety or gesture classes that are similar in appearance. Although this is a limitation for deployment in real-time, findings reveal the importance of designing human-noise datasets that are deployment-oriented. This work describes the strengths and limitations of attention-based models and provides insights for designing a more robust framework for assistive communication applications.

International Journal of Scientific Engineering and Research
Panjab University (IN)
Hand Gesture Recognition Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Analysis of Vision Transformers for American Sign Language Recognition in Unconstrained Visual Conditions — Sumit Budhiraja, Sunil Agrawal, et al. · International Journal of Scientific Engineering and Research (2026) | TGRS Research Map | TGRS