Adaptive Spatiotemporal Feature Fusion for Visual Object Tracking

Historical information is crucial for resolving the challenges of the target state changes. Plenty of methods following memory network, which map the historical data and search areas into a unified embedding space to solve feature misalignment, have attained great performance in visual tracking. However, single-scale feature fusion of historical prompts and spatial features, without fully utilizing the rich information embedded in the prompt, struggles to handle complex scenarios such as low resolution and fast motion. To address the above problem, we propose the Adaptive Spatio-Temporal Information Fusion Module, which dynamically integrates spatio-temporal information from various embedding spaces during the inference process of each frame, unlocking the potential of the prompt. Additionally, to alleviate the issues of semantic bias in historical prompts, we propose Adaptive Triplet Attention to refine historical data through cross-dimensional interactions between channels. In the end, we built a novel tracker called ASTrack, which adaptively generates the task-relevant discriminative features through the fusion of spatio-temporal features. Our method is evaluated on six tracking benchmarks, and the results confirm the effectiveness and robustness of the proposed approach.