Expert-Driven Frequency-Domain Modeling for Video Moment Retrieval

Video moment retrieval (VMR) aims to localize the temporal segment in an untrimmed video that semantically matches a natural-language query. Although recent detection transformer (DETR) approaches have improved this task, they primarily emphasize query-conditioned cross-modal alignment and often underexploit the intrinsic temporal dynamics of videos. In particular, clip-level features contain frequency-domain cues that reveal how visual content changes across time, yet such cues remain underexplored for moment localization. In this paper, we propose FE-DETR, a frequency-domain expert-driven DETR framework for dynamic-aware video moment retrieval. FE-DETR introduces Multi-span Frequency Experts (MFE), which select expert-specific span lengths from frequency-domain scores of a video and perform expert-specific frequency decomposition to capture multi-scale dynamic patterns. The resulting frequency-enhanced representation is fused with regular cross-modal features for query-video alignment. To further improve localization, we design an Expert-targeted Encoder (ETE) with a distinction loss that encourages query-relevant and background clips to be handled by different expert groups, and we incorporate a multi-expert decoder for fine-grained temporal boundary prediction. On TACoS, FE-DETR achieves a three-seed mean of 40.37% Recall@1 at an intersection-over-union (IoU) threshold of 0.5 and a mean IoU of 37.52%. Experiments demonstrate strong performance on TACoS and competitive results on QVHighlights and Charades-STA.