RGB-T object tracking method based on multi-granularity adaptive fusion
RGB-T tracking exploits the complementary modalities of RGB and thermal infrared (TIR) to achieve robust performance under challenging conditions, such as low-light or adverse weather. While recent Transformer-based trackers model cross-modal interactions effectively, they often rely on fixed-resolution features, which limits their adaptability to large variations in target scale. Moreover, naive fusion of Convolutional Neural Network (CNN) and Transformer features may introduce representation discrepancies and degrade tracking performance. To address these issues, we propose MGAFTracker, a novel RGB-T tracker based on Multi-Granularity Adaptive Fusion. The proposed MGAFTracker consists of two key components: (1) a Multi-Granularity Feature Pyramid (MGFP) module that extracts and refines hierarchical CNN features across multiple spatial scales and receptive fields to construct multi-granularity representations, and (2) a Cross-Domain Feature Recalibration (CDFR) module that dynamically recalibrates CNN features through channel-wise modulation weights derived from Transformer features, enabling adaptive fusion within the Transformer encoder. Extensive experiments on two public RGB-T tracking benchmarks demonstrate the effectiveness of MGAFTracker.
Authors
- Wenxu Liu (ORCID: https://orcid.org/0000-0002-1647-9162)
- Qiaona Zheng (ORCID: https://orcid.org/0000-0003-2615-4801)
- Aihua Liu
- Songjiang Feng
- Tao Wu
Institutions
- Ministry of Education of the People's Republic of China (CN)
- China Mobile (China) (CN)
- China Electronics Technology Group Corporation (CN)
- HBIS (China) (CN)
- Space Engineering University (CN)
Publication Details
- Journal
- Scientific Reports
- Published
- 2026-09-12
- DOI
- https://doi.org/10.1038/s41598-026-67207-4
- Primary Topic
- Video Surveillance and Tracking Methods
- Type
- article
- Field-Weighted Citation Impact
- 0.00