Video Stabilization Based on Self-Attention Structure with Temporal Memory

In recent years, video capture has expanded to diverse environments; however, recording video with unstable or heavily shaking devices frequently results in motion-induced disturbances that significantly degrade visual quality. To address this issue, video stabilization techniques have increasingly leveraged the powerful feature extraction capabilities of neural networks to eliminate unwanted motion disturbances while preserving visual content integrity. However, stabilized videos often exhibit artifacts and distortions because current neural network architectures cannot effectively incorporate temporal features, resulting in structural inconsistencies across frames. Moreover, existing deep learning methods frequently require optical flow calculations for input videos, adding substantial computational overhead. To overcome these challenges, this paper proposes a video stabilization method based on self-attention architecture with temporal memory. Our model integrates a self-attention mechanism with a ConvLSTM (Convolutional Long Short-Term Memory) network, which significantly enhances the network's ability to capture temporal relationships between consecutive frames, thereby more effectively eliminating motion disturbances. Additionally, our loss function design incorporates the movement of corresponding feature points across video frames to calculate acceleration and velocity changes, enabling more precise correction of motion disturbances based on feature point trajectory variations. The proposed network outperforms existing methods across all video stabilization assessment metrics, achieving a crop rate of up to 91.6% while reducing the parameter count by 63%, demonstrating both superior stabilization quality and computational efficiency.