Action-Responsive Contrastive Network for Fine-Grained Skeleton-Based Action Recognition

Currently, fine-grained skeleton action recognition based on graph convolutional networks (GCNs) has become an important research focus. Fine-grained action recognition refers to the accurate recognition of subtle, complex or detailed actions. This task is particularly challenging due to the limited appearance information in skeleton data and the limitations of predefined single-topology skeleton structures. To address these challenges, we propose an action-responsive contrastive network (ARCN). The network consists of two main components: an action-responsive graph convolutional network (ARGCN) with enhanced skeleton topology and a fine-grained action comparator (FAC) that uses feature contrastive learning to explore the latent space of motion features. The ARGCN contains two specialized modules: the action-responsive topology (ART) module, which captures important motion features through the learned action-specific topology structure matrix and multiscale temporal features; and the action-responsive attention (ARA) module, which learns complex spatiotemporal skeleton attention information. These modules jointly generate a multichannel cross-temporal dynamic skeleton joint attention topology map tailored for the specific action being analysed. To further clarify the fine-grained action feature differences, the FAC is integrated in some stages of the ARGCN. The FAC performs spatiotemporal decoupling of feature maps, classifies and contrasts similar and different fine-grained motion features, and builds a learnable latent space for fine-grained motion, thereby improving classification performance. Our model is evaluated on six public datasets: NTU RGB+D, NTU RGB+D 120, NW-UCLA, UAV-Human, Finegym, and Diving48. It achieves 91.2% accuracy on the NTU RGB+D 120 dataset X-Set, 97.2% accuracy on the NW-UCLA dataset, 44.6% accuracy on the UAV-Human dataset CSv1, 72.0% accuracy on the UAV-Human dataset CSv2, 95.3% accuracy on the Finegym dataset, and 54.3% accuracy on the Diving48 dataset, which are competitive results compared with the state-of-the-art methods.