Learning Convolutional Representation of Cross-Spatial Multiscale Graphs for Human Skeleton Action Recognition

Human skeleton can be classified as a form of graph-structured data. GCNs have gained popularity in human skeleton action recognition due to their exceptional capacity to handle graph data. Human action is characterised by complexity and multi-scale variation. For example, “clapping” is a localized action, while “jumping” is a global action. Thus, the important issue in GCN-based human action recognition (HAR) is the achievement of efficient multi-scale modelling. However, current GCN techniques focus solely on spatial modeling through the use of a fixed or learned adjacency matrix. This methodology hinders the acquisition of multi-scale characteristics of human actions, hence impacting the model's recognition accuracy. To address the aforementioned issues, this work introduces a cross-spatial multi-scale graph convolution (CSM-GCN) as a solution. CSM-GCN handles multi-scale spatial information by adding a cross-spatial multi-scale attention block (CMAB) to the adaptive graph convolution. In addition, we introduce the Inter-Frame Attention Block (IF A) to localize the key frames of the action sequence, which makes the model more flexible and effective in temporal modeling approach. Comprehensive experiments are conducted to evaluate CSM-GCN in the publicly available datasets NTU RGB+D and NTU RGB+D120 for human skeleton action recognition. The experimental results show that CSM-GCN achieves higher recognition accuracy with lower number of computational parameters, which is competitive with the state-of-the-art action recognition methods.