Improving Object Tracking Performances With Frequency Learning for Event Cameras

Due to their characteristics of low latency, high temporal resolution, and wide dynamic range, event cameras have emerged as advanced visual sensors for robust multi-modal tracking. However, recent approaches for processing multi-modal sensor data often rely on multiple disjointed modules, such as feature extraction, fusion, and adaptive learning, resulting in high model complexity. Moreover, the feature representations of sensor data obtained through these methods are limited by the traditional attention mechanism, which overlooks the role of frequency domain knowledge in enhancing modality-specific features. This work introduces a unified tracking framework named Frequency-Aware Event Frame Tracker (FAEFTrack) to reduce model complexity and improve modality-specific features. Specifically, a voxel-based event representation is adopted to effectively capture spatial-temporal details from event data. Based on the vision transformer network, we design a single-stream sensor data fusion backbone which is able to integrate feature learning with relation modeling and simplify the tracking process. Furthermore, a frequency attention block with multi-frequency attention is proposed to enhance modality-specific features in the frequency domain. Extensive experiments on the VisEvent and COESOT datasets demonstrate that our approach outperforms state-of-the-art methods, achieving improvements of 3.6%/1.5% and 1.6%/4.1% in SR/PR scores, respectively.