Leveraging Mamba for Enhanced Multimodal Few-Shot Action Recognition

Few-Shot Action Recognition (FSAR) widely use in Action Recognition domains with limited labeled data, such as medical surgery. Existing FSAR methods often rely on single-modal approaches, which suffer from limited feature representation and a lack of semantic understanding. To overcome these limitations, we propose MoMA, a novel Mamba-based multimodal FSAR framework, designed to optimize the learning of multimodal information and improve the performance of few-shot action recognition tasks. MoMA integrates a multimodal large language model (MLLM) to generate semantically rich representations from both textual and video data, which are then aligned and fused with visual features. To address the challenges of domain-specific prior knowledge and intra-class variance, we introduce a dual-branch contrastive learning architecture that reinforces category distinctions and mitigates the accumulation of intra-class heterogeneity. Our experimental results show that MoMA outperforms state-of-the-art methods on the HMDB51, Kinetics, and SSv2 datasets, achieving improvements of 1.5%, 1.9%, and 3.5%, respectively, in the 1-shot setting.