mmDigit: A Real-Time Digit Recognition Framework in Air-Writing Using FMCW Radar

Millimeter-wave (mmWave) radar sensors show significant promise in noncontact human-computer interaction. Using air-writing as a substitute for conventional input devices, such as keyboards and mice has become a pivotal topic in contemporary research. In response to the absence of a dataset in existing studies about air-writing digits and the insufficient exploration of real-time recognition in edge devices, we propose a real-time air-writing digit recognition framework based on mmWave radar, termed mmDigit. Initially, we use mmWave radar equipped with frequency-modulated continuous wave (FMCW) technology to collect digital echo data and design a data processing pipeline to track and reconstruct digital trajectory images. These images are subsequently fed into a lightweight neural network, which is only 6.9K in parameter size, for exploring the images’ quality and the recognition and cross-user capabilities of small-scale air-writing datasets. To enhance mmDigit’s performance, we implement a transfer learning strategy to accommodate a broader range of digit writing styles and habits, achieving a recognition accuracy of 99.14% and a cross-user capability of 94.13%. Additionally, applying a knowledge distillation strategy enables the lightweight network to extract and learn deep-layer features, thereby improving the cross-user recognition accuracy to 96.22%.