Cross-domain gesture activity recognition via dual-path prototypical network using Wi-Fi signals

Abstract With the rapid advancements in human-computer interaction, gesture activity recognition based on Wi-Fi sensing has emerged as a prominent research area in artificial intelligence. However, cross-domain gesture activity recognition faces significant challenges due to the differences in data distribution between domains. Specifically, models trained in the source domain often experience a significant drop in recognition performance when applied to the target domain. To address this issue, this paper proposes a cross-domain recognition method based on a dual-path prototypical network, FAPNet (Fusion of Amplitude and Phase Network). Compared to traditional single-path prototypical networks, which are prone to information loss due to single-modal feature extraction, and fixed-weight amplitude-phase feature fusion strategies, which struggle to adapt to the nonlinear shifts in cross-domain feature distributions, the proposed method constructs a dual-stream heterogeneous learning architecture that significantly enhances cross-domain recognition performance. Specifically, FAPNet extracts amplitude and phase features through independent branches and further enhances key gesture representations by incorporating a multi-scale spatiotemporal enhancement module; building on this, this paper designs a hierarchical prototype optimization mechanism, which generates global prototypes through self-supervised clustering and incorporates an intra-class feature cosine weighting strategy to effectively enhance the domain adaptability of prototype representations; furthermore, to achieve adaptive fusion of amplitude and phase features, this paper constructs a dual-metric mechanism that evaluates modality uncertainty through feature distance correlation analysis, enabling the autonomous adaptation of amplitude-phase weights. Experiments show that the proposed method achieves high classification accuracy in intra-domain tasks and demonstrates strong cross-domain recognition performance when training data is sufficient or cross-domain samples are limited. Compared to the cross-scenario performance of single-path prototypical networks, it improves by 2.66%, providing an effective solution for cross-domain gesture recognition.