Upright and Inverted Motion-enhanced Network for Video Action Recognition

Human action recognition in videos has achieved significant research progress. However, existing methods lack effective and concise representations and feature extraction approaches for frame sequences. Most of these methods use the entire sequence as input, which can easily introduce noise and adversely affect computational efficiency. Additionally, current methodologies are deficient in exploring diverse ensemble strategies, typically relying solely on the inference from the original upright viewpoint and neglecting the exploration of other perspectives. Improving video frame feature representation and extraction approaches, as well as exploring new ensemble strategies remain open challenges. To address these issues, we propose a human-centric preprocessing method that represents video frame sequences as human posture variation images. Meanwhile, we introduce a novel ensemble strategy combining normal and reversed posture features to leverage complementary information from these different perspectives. Finally, we propose Upright and Inverted Motion-enhanced Network (UI-MEN) which integrates motion information and posture variation representations in early stages, and fuses upright and inverted features for final decision. Through extensive experiments on NTU RGB+D and NTU RGB+D 120 datasets, our modal shows promising results compared with state-of-the-art methods while maintaining low computational complexity.