FCI-DETR: frequency-context interaction transformer for RGB-IR object detection

Abstract RGB-infrared (RGB-IR) object detection combines visible appearance cues with infrared thermal information and supports perception under complex illumination, weak texture, occlusion, and large scale variation. However, existing fusion detectors still face three coupled problems: modality-specific structures may be compressed before fusion, unreliable cross-modal responses may be amplified during fusion, and infrared context may be diluted during multi-scale feature propagation. To address these problems, this paper proposes FCI-DETR, a frequency-context interaction Transformer that formulates RGB-IR detection as a staged adaptive modality-control process. First, to alleviate modality-specific structure loss before fusion, the Lightweight Frequency Adapter (LFA) preserves spatial-frequency cues through spatial, Haar frequency, and identity experts. Second, to suppress unreliable cross-modal responses during fusion, the Cross-modal Feature Interaction Module (CFIM) calibrates dual-modal features through adaptive global modality gating, local complementary compensation, and channel-sequence correction. Third, to reduce infrared context dilution after fusion, Contextual Kernel-Guided Fusion (CKGF) injects infrared features as conditional context during top-down and bottom-up multi-scale propagation. Experiments were performed on four public RGB-IR datasets, including M3FD, DroneVehicle, VEDAI, and FLIR. FCI-DETR achieved mAP 50 scores of 88.6%, 84.3%, 86.7%, and 81.5% on these datasets, respectively, achieving competitive performance across diverse RGB-IR detection scenarios.