UbiGest: Smartphone-Based Ubiquitous Gesture Recognition With Wi-Fi

Wi-Fi channel state information (CSI)-based gesture recognition systems have recently gained significant attention. One major challenge faced by these systems is that CSI measurements are affected by various conditions, including environmental factors and user orientation, leading to degraded sensing performance in unseen environments. Due to this challenge, most existing systems require multiple transceivers placed only a few meters apart from each other, and users are required to perform gestures within the proximity of these transceivers. Despite these systems robustly sensing users, their sensing range and mobility are severely restricted. In this article, we propose UbiGest, a smartphone-based gesture recognition system. UbiGest leverages beacon frames from multiple nearby APs over adjacent channels, enabling ubiquitous gesture recognition while mitigating the effects of the environment, user position, and user orientation. The system, even with limited computational resources, extracts features from noisy CSI data collected from a smartphone and robustly classifies a user’s gesture. By incorporating few-shot learning, the system maintains its robustness in challenging conditions, such as in entirely new environments with only one or two available APs. The extensive experimental results demonstrate that UbiGest achieves an accuracy of 95.3% in in-domain evaluation and 92.6% in cross-domain evaluation, on average.