Learning Improved Dense Trajectories in Guided Feature Space

Video trajectories offer a compact representation for video-based tasks. However, in real-world videos, dense trajectories often suffer from spatial and temporal distortions caused by camera motion, scale changes, and perspective shifts. These distortions can lead to blurring and viewpoint variations. To address the identified limitations, a novel architectural framework is introduced that accurately determines dense motion trajectories through a guided feature representation. By incorporating external supervision to steer deformable convolution layers, our model demonstrates enhanced resilience to geometric distortions. Additionally, a temporal warping technique establishes correspondences between consecutive frames, allowing us to incorporate less distorted information. The proposed architecture, consisting of temporal warping and a guided LCDC module, demonstrates superior performance on action recognition and video captioning tasks when evaluated on UCF-101 and MSVD datasets. The experimental results and an ablation study shows that the proposed architecture, which outperforms the existing methods with 1.4% accuracy on 50 salad dataset.