Language-Guided Test-time Prompt Tuning for Zero-shot Skeleton-based Action Recognition

Existing Zero-shot Skeleton-based Action Recognition (ZSAR) methods primarily concentrate on aligning skeleton features with semantic embeddings, neglecting the impact of fine-tuning strategies on domain-specific behaviors, which can undermine the zero-shot generalization. To this end, we propose a novel Language-Guided Test-time Prompt Tuning (LG-TPT) method for the ZSRA task, which comprises two key modules: Language-Guided Prompt Generator (LGPG) and Language-Guided Multi-Features Augmentation (LG-MFA). Specifically, LGPG stabilizes prompts through linguistic guidance, enhancing generalization by using skeleton features as offsets. LG-MFA enriches data diversity by simulating noise, masking, and integrating text embeddings. Moreover, we introduce a Confidence Weighted Selection (CWS) strategy to prioritize high-confidence predictions while discarding low-confidence views. Our experiments demonstrate that LG-TPT significantly outperforms existing methods in recognizing unseen actions on three public datasets.