LViTE: A Lightweight Vision Transformer with Ensemble Classification for Sign Language Recognition

Sign language recognition is essential for human-machine interaction, supporting communication for individuals with hearing and speech impairments. However, challenges remain due to variability in hand shapes, orientations, motion dynamics, and environmental factors such as lighting and occlusion. Moreover, many existing models are computationally intensive, limiting their applicability in resource-constrained settings. This paper introduces the Lightweight Vision Transformer with Ensemble Classification (LViTE), a streamlined framework that balances accuracy and efficiency. LViTE employs a reduced Vision Transformer backbone with fewer encoder layers and attention heads to lower computational cost, while an ensemble-based classification mechanism enhances robustness through aggregated predictions from multiple decision trees. Evaluated on three benchmark datasets—American Sign Language (ASL), ASL with Digits, and NUS Hand Posture—LViTE achieves state-of-the-art accuracies of 99.98%, 99.98%, and 99.97%, respectively. These results demonstrate LViTE’s effectiveness and suitability for real-time deployment in human-machine systems where both performance and efficiency are critical.