Spatial-Temporal Transformer with Contrastive Learning for Skeleton-Based Human Action Recognition

Skeleton-based action recognition is becoming increasingly important since it offers fundamental understanding for applications ranging from human-computer interaction to surveillance. However, labeling skeleton data manually for training models is costly. As a result, contrastive learning has emerged as a promising solution, significantly enhancing the model's ability to recognize human activities without relying on extensive annotated datasets. Pre-training models using contrastive learning emphasizes the potential for efficient learning and enhanced performance. This paper proposes a method that incorporates Transformers into the contrastive learning framework as the encoder. This encoder is essential for converting skeletal input into discriminative representations for accurate recognition. Deviating from conventional supervised learning techniques, our approach exploits the advantages of Transformers to effectively capture the complex spatial-temporal correlations inherent in skeletal data. Experimental results on benchmark datasets demonstrate the efficiency and effectiveness of our Spatial-Temporal Transformer encoder. Our findings indicate the significant advantages of integrating sophisticated encoding methods with contrastive learning, opening the door to less data-dependent annotation for action recognition.