MagNeura: DNN Factored Magnetic Sensor Based Finger Tracking Wearable Design

Finger pose tracking is an important task in various healthcare domains, including rehabilitation. Existing solutions based on computer vision, strain gauges, Inertial Motion Units, thermal cameras and surface electromyography pose a variety of limitations. This paper proposes MagNeura, a compact system that estimates finger angles based on tri-axial magnetometers. Unlike conventional mathematical model-based mapping between sensor data and finger angles, the sensor data is fed to a deep neural network (DNN) to estimate the angles. The system is easily calibrated to different users and environments. Moreover, the setup remains independent of the individual’s physiological signals, which otherwise limits the scalability and generalizability of the usage. The proposed wearable system is characterized for precise finger movement, and a deeper analysis of the system’s accuracy over the ground-truth results is presented. The proposed wearable system enables tracking of individual and multiple fingers to estimate various hand poses which otherwise remains a challenge. Real-time tracking of dynamically changing finger gestures has various use cases including physical rehabilitation, human-computer interaction, gaming, and sign-language translation. The precise finger tracking established from AI applied on magnetic sensing is a step towards developing a reliable and efficient body motion tracking system.