UWB Impulse Radar-Based Open-Set Gesture Recognition Using Transformer and One-Versus-Rest Classifier

Gesture recognition can provide a natural and user-friendly interface to a variety of systems. However, in order for the real-world applications to apply gesture recognition as a tool for human-computer interaction (HCI), it is necessary not only to accurately classify predefined gestures but also to correctly identify undefined gestures. In this article, we design a novel open-set gesture recognition (OSR) system which leverages ultra-wideband (UWB) impulse radar-based wireless sensing and deep learning technology to effectively detect both known and unknown gestures. In the proposed system, the OSR task is decomposed to two steps of closed-set classification and deep inspection. The first closed-set classification step extracts the features from the channel impulse response (CIR) matrix of UWB radar and predicts one of the known gestures. The second deep inspection step investigates more deeply whether the predicted known gesture is correct or an unknown gesture. The feature extractor is composed of two Transformers, which extract temporal features and spatial features unique to each gesture from the CIR matrix, respectively. The second stage consists of multiple binary deep inspectors, each of which is assigned to one of the known gestures, and this method not only greatly simplifies the training process of deep inspection but also remarkably enhances the OSR performance. Extensive experimental results conducted on a public dataset demonstrate that our system significantly outperforms the state-of-the-art OSR systems.