Motion-Consistent Representation Learning for UAV-Based Action Recognition

Action recognition aims to identify action categories in trimmed videos captured by multimedia devices, which often suffer from jitter, especially in uncrewed aerial vehicle (UAV) applications. Existing methods typically ignore the effect of jitter on actor motion or rely on external stabilization tools trained on large-scale unstable video datasets that may not be tailored to specific tasks. To address this, we propose the Stabilization-enhanced Recognition Network (StaRNet), an end-to-end framework that integrates video stabilization and contrastive learning. Inspired by traditional stabilizers, StaRNet’s Motion-aware Stabilization Module (MSM) constructs positive and negative video pairs to model instability: positive pairs use optical flow to estimate frame motion and refine rigid motion via keyframe estimation for motion-aware stabilization, while negative pairs assess temporal consistency using motion cues to boost classification. Moreover, we introduce a Motion-aware Constraint (MC) that regulates dynamic stabilization to adapt to varying motion patterns and enrich action representations. Experiments on UAV benchmarks show that StaRNet outperforms state-of-the-art methods and substantially enhances video stabilization. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/lwxfight/-StaRNet</uri>