S2AT-GCN: A Spatial Self-Attention Temporal Graph Convolutional Network for Gait-Based Person Recognition

Graph convolutional networks (GCNs) excel in skeleton-based action recognition by modelling human body skeletons as spatiotemporal graphs. In current GCN-based approaches, the graph topology is manually defined and fixed across all layers and input samples. There may be better approaches for hierarchical GCN and gait recognition data. Using spatial-temporal GCN (ST-GCN) and multi-order adjacency matrices have common limitations in keeping long-range dependency between physically disjoint joints. The multi-order adjacency matrix shows the local bias problem, and ST-GCN shows the static spatial partitioning problem. To address these limitations, in this work, a Spatial Self-Attention (S2A) module with a fully connected graph convolutional (GC) operator is introduced to adaptively capture the dependencies between the joints that are physically apart and neglect the multi-scale adjacency matrix. Here, the S2A module understands the intra-frame interactions between different body parts and joints. The temporal convolutional module is utilized to analyze the correlations between joint frames. The spatial and temporal model captures both long-range dependence of joints and node temporal behaviours. The model shows remarkable accuracy of $\mathbf{9 6. 9 2 \%}, \mathbf{9 0. 4 2 \%}$, and $\mathbf{9 2. 8 2 \%}$ on the CASIA-B dataset at normal, clothing, and bag-carrying conditions, respectively. This is a significant improvement of $0.92 \%, 1.52 \%$, and $0.42 \%$ for normal, carrying bag, and clothing conditions compared to the existing methods presented at Table III.