Attention-Guided and Topology-Enhanced Shift Graph Convolutional Network for Skeleton-Based Action Recognition

Graph Convolutional Networks (GCNs) have emerged as a game-changer in skeleton-based action recognition. However, most previous works are resource-heavy, with large FLoating-number OPerations (FLOPs) limiting the model’s potential. A recent work involving shift operators to GCN (Shift-GCN) has successfully introduced a lightweight GCN, but there is still a performance gap compared to previous results. Inspired by Shift-GCN, we propose an innovative and novel model named attention-guided and topology-enhanced shift graph convolutional network (AT-Shift-GCN), which continues the lightweight benchmark and provides a more powerful performance. We employ a topological transfer operation to aggregate the information flow of different channels and extract spatial information. In addition, to extract temporal information across scales, we apply attention to interacting with shift convolution kernels of different lengths. Furthermore, we integrate an ultralight spatiotemporal attention module to fuse spatiotemporal details and provide robust neighborhood representation. In summary, AT-Shift-GCN is a breakthrough in skeleton-based action recognition that provides a lightweight model with enhanced performance on three datasets.