Real-Time Sign Language Interpretation via Customized Sign Language Gloves and Motion Retrieval

A sign language interpretation system aims to translate sign gestures into spoken or written language in real time, enabling signers and non-signers to communicate in their familiar linguistic forms. However, vision-based approaches suffer from hand occlusion, lighting variability, and complex backgrounds, while Deep Neural Network (DNN)-based methods incur heavy computational costs that hinder real-time use on resource-constrained platforms. In this paper, we propose sign language gloves and a lightweight motion retrieval method for real-time sign language interpretation that runs on mobile devices and embedded systems. The sign language gloves integrate flex sensors, an inertial measurement unit (IMU), and pressure sensors to accurately capture gesture features, including finger bending angles, hand orientation, movement trajectories, and fingertip contacts with body parts, enabling recognition of touch-based gestures. For the motion retrieval method, we build a comprehensive gesture dataset with the gloves and perform feature analysis on each sign language gesture to avoid redundant information in the dataset. During interpretation, our system employs a feature-labeling mechanism to ensure gesture distinguishability and a gesture retrieval algorithm to evaluate movement continuity and similarity. This allows the system to identify corresponding feature labels and consolidate them into complete sign language vocabulary entries. The proposed motion retrieval method is characterized by low computational complexity and a well-defined data structure. This makes it suitable for integration into embedded systems, offering real-time performance and high portability for practical deployment. In our experiments, the proposed system achieved an average recognition accuracy of 92% on a gesture dataset covering 300 sign language words.