Synergizing Acoustic and Wi-Fi Signals for Device-Free Gesture Recognition

Gesture recognition has significant applications in areas such as assisted living, e-health, and human-device interactions. Moving beyond conventional computer vision techniques, recent studies have increasingly adopted ubiquitous methods like Wi-Fi and acoustic signals, which provide a cost-effective solution for device deployment. In this paper, we explore these two ubiquitous techniques to enhance gesture recognition, focusing on overcoming challenges associated with multi-modal fusion. To harmonize information from these inherently different signal types, we propose a tailored fusion strategy specifically designed for Wi-Fi and acoustic signals. Traditional multi-modal fusion methods often lack a theoretical framework due to insufficient analysis of the fundamental characteristics of different signals. To address this gap, we introduce the concept of the Hybrid Zone, a novel theoretical framework that models the interaction and fusion of acoustic and Wi-Fi sensing signals. The Hybrid Zone offers a unified perspective on the interaction between acoustic and Wi-Fi sensing areas and delivers insights into the granular synthesis of their velocity profiles. Our experimental results demonstrate strong performance, achieving a gesture recognition accuracy rate of 94.69%.