mmWave Radar-Based Unsupervised Gesture Recognition via Image-Aligned Heterogeneous Domain Transfer

Human Gesture Recognition (HGR) using mmWave radar has become increasingly promising due to its exceptional contactless perception sensitivity. Conventional approaches predominantly rely on supervised models to learn radar signals, thus incurring substantial costs associated with annotation. To address this limitation, certain works embrace transfer learning to effectively transfer knowledge from labeled source domain to unlabeled target domain, achieving unsupervised recognition in the target domain. However, existing transfer-based methods still necessitate large-scale labeled source domain radar data, thereby constraining their practical applicability. To this end, we propose a novel unsupervised solution for mmWave-based HGR by transferring public image gestures to radar data, eliminating the need for acquiring labeled radar data in source domain. We aim to establish heterogeneous alignment between images and radar signals, facilitating cross-domain transfer. Initially, we mitigate the negative impact of data heterogeneity by employing sophisticated signal processing techniques to convert raw radar signals into gesture trajectories. Subsequently, we introduce an Adversarial-Contrastive Domain Transfer Model (ACDTM) to achieve fine-grained alignment. ACDTM not only confuses the source and target domains by adversarial learning, enabling the acquisition of domain-invariant features, but also designs a robust similarity matrix to facilitate intra-class alignment through contrastive learning. Additionally, ACDTM conducts adversarial self-training on target domain with pseudo-labeled distribution. Our experimental findings substantiate that the unsupervised accuracy achieves about 80<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>92% on different mmWave gesture datasets, outperforming existing unsupervised HGR schemes by large margins. Code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/onlinehuazai/mmGesture</uri>.