Towards 4D Gesture Reconstruction Through Leaky Acoustic Waves

In-air gestures play a crucial role in many emerging applications – from smart homes where users can adjust lighting [1] or temperature settings [2] with simple hand gestures or finger motions, to virtual reality (VR) scenarios, allowing users to interact with virtual objects without physical touch. In-air gestures also enable seamless interaction in healthcare, maintain reliability when wearing gloves, and enhance comfort by eliminating physical contact, making them a versatile complement to traditional touch-based controls. Most wireless sensing systems designed for in-air gesture control treat gestures as single actions [3] such as recognizing whether it is a pinch, a sweep, or a friction gesture. However, they overlook the critical details of gesture motion such as the speed or intensity variation (i.e., how the user performs gestures), which are crucial for more realistic gesture interactions. For instance, if the music volume is too low, the mobile user often turns the knob rapidly to raise the volume to a reasonable level, then slows down to fine-tune it to the ideal setting. Therefore, obtaining the speed variations of a friction gesture (emulating the way of turning the volume knob) could allow users to control volume dynamics more precisely and realistically.