MSCF-Net: Multi-stage Cascaded Fusion Network for Early Action Prediction

Early action prediction (EAP) aims to infer action categories from partially observed sequences, a capability critical for real-time applications such as human-computer interaction and intelligent surveillance. This task faces inherent challenges of incomplete information and indiscernible discriminative features in early stages. To address these issues, we propose MSCF-Net, a novel skeleton-based Multi-stage Cascaded Fusion Network featuring two key innovations:(1) A cascaded multi-scale feature extraction architecture that progressively downsamples sequential data to hierarchically capture action semantics across temporal scales, significantly enhancing sensitivity to subtle early cues;(2) A partition-specific skeleton-temporal self-attention mechanism (JTPSA) that decomposes joints and frames into predefined relationship types (e.g., local joint motions vs. global limb coordination) and computes attention independently within each partition. This design enables precise modeling of spatio-temporal dependencies while reducing computational complexity.Extensive experiments on public benchmarks demonstrate that MSCF-Net achieves significant and consistent performance improvements over state-of-the-art methods in early action prediction, particularly under low observation ratios.