Research on Multimodal Gait Recognition Based on Silhouette and Skeleton

To address the issue of suboptimal fusion between silhouette and skeleton features in the GaitMix model that degrades gait recognition accuracy, this paper proposes a novel silhouette-skeleton-based multimodal gait recognition method (GaitMHA). First, skeleton data is decoupled into bone and joint components for separate feature extraction, enhancing the model's capability to process heterogeneous modalities. Second, silhouette, bone, and joint features are fused via a Block-wise Self-Attention module, enabling adaptive weight allocation across modalities to extract integrated features more effectively. Finally, a Gating Module (G-Module) is introduced to mitigate overfitting and gradient explosion risks associated with block-wise self-attention networks, further enhancing model robustness. Experiments on the CASIA-B gait dataset demonstrate that the proposed method achieves average recognition rates of 98.30% (normal walking, NM), 95.90% (carrying a bag, BG), and 89.20% (wearing a coat, CL), outperforming existing gait recognition methods such as GaitPart, CSTL, GaitGL, and GaitMix. The results validate the efficacy of GaitMHA in improving fusion performance across silhouette, bone, and joint modalities.