SpikingMotionFormer: a neuromorphic transformer architecture for low-power recognition of traditional ethnic sports movements

Abstract Traditional ethnic sports constitute a vital component of intangible cultural heritage. Their digital preservation and automated analysis pose significant challenges, particularly in achieving high recognition accuracy under strict computational and energy constraints on edge devices. To address these challenges, we propose SpikingMotionFormer , a novel neuromorphic transformer architecture for efficient and accurate recognition of traditional ethnic sports movements. SpikingMotionFormer tightly integrates spiking neural networks (SNNs) with transformer-based attention mechanisms, and incorporates an adaptive spiking threshold strategy that dynamically adjusts neuronal firing behavior according to input statistics, improving both representational capacity and energy efficiency. To capture complementary motion cues, we further design a hierarchical attention-based multimodal fusion strategy that effectively combines inertial signals (accelerometers and gyroscopes) with visual information. To alleviate the reliance on large-scale labeled datasets, we introduce a self-supervised learning framework that explicitly exploits the intrinsic structure of motion data. The framework jointly leverages temporal contrastive learning to model short-term motion continuity and a cross-modal motion consistency objective to align representations across heterogeneous sensors, enabling the model to learn robust and discriminative motion representations from unlabeled data before supervised fine-tuning. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed approach. SpikingMotionFormer achieves up to 99.85% accuracy while reducing inference latency and energy-related costs by up to 53% compared with conventional transformer-based models, highlighting its suitability for resource-constrained deployment scenarios.