Adaptive Part-Level Embedding GCN: Toward Robust Skeleton-Based One-Shot Action Recognition

This article explores the problem of skeleton-based one-shot action recognition (SOAR), aiming to build a high-performance action classifier using only a single reference skeleton sequence per action. The key motivation is to develop an embedding space that can be generalized from previously learned, data-abundant actions to novel, data-scarce ones. Nevertheless, this task becomes considerably challenging when the skeleton data are disrupted through occlusions, a common occurrence in real-world applications. This work proposes a novel adaptive part-level embedding graph convolutional network (APLE-GCN), which leverages multistream data streams to extract joint-level features and adaptively aggregate them into part-level embeddings. We go beyond traditional part-based models by not adhering to a fixed strategy of dividing the skeleton into body parts. Instead, we develop a data-driven method to learn instance-specific part templates and compute part-level embeddings based on the correlation between joint-level features and these templates. Unlike body parts defined by skeleton topology, parts extracted via the proposed method, referred to as semantic parts, are intended to capture characteristic motion patterns of various actions and offer increased flexibility when dealing with occlusions. Extensive experiments conducted on five benchmarks demonstrate that the embeddings of semantic parts exhibit strong generalization ability, bringing our model to the state-of-the-art.