FP-GCN: A Novel Feature Pyramid Graph Convolutional Network For Skeleton-based Action Recognition

For skeleton-based action recognition, the aggregation of features among human skeletal joints is a critical factor, which influences recognition accuracy in graph convolutional networks. Existing methods often neglect the extraction of skeletal structure features at different scales, which limits the ability of the model to understand actions. To address this issue, we propose a novel Feature Pyramid Graph Convolutional Network(FP-GCN) that enhances the representational capability of the model by capturing the multi-scale spatial features of the skeleton sequence. In detail, we propose an attention-based graph pooling module that effectively contracts the skeleton to multiple lower-order sub-graphs, which serve as spatial representations of the skeleton at corresponding levels. The original skeleton and these sub-graphs are combined to form the feature pyramid, where joints of each level span in the same semantic space. Additionally, we introduce a graph unpooling module to restore the pooled sub-graphs to their original topology. Moreover, we adopt a multi-loss strategy across different spatial scales, encouraging the model to learn more comprehensive skeletal features. Finally, we validate our proposed model on three large-scale datasets, achieving the highest accuracy compared to state-of-the-art methods. We conduct numerous comparative experiments to verify the effectiveness of modules.