Explicit temporal inductive biases for video moment retrieval via multi-scale Gaussian priors

Video moment retrieval aims to localize video segments corresponding to natural language queries. While Transformer models are widely applied, their localization performance declines when processing diverse temporal scales. This limitation arises from the dependence on implicitly learned decoder queries. Without explicit temporal grounding, these unstructured embeddings lead the decoder to perform a global search across the full video sequence. We propose Gaussian-DETR to address this by introducing explicit temporal awareness into the decoding process. Our framework replaces unstructured embeddings with queries constructed from multi-scale Gaussian temporal priors, parameterized by a learnable center and scale. To adapt these priors to the input, we design a multi-scale Gaussian pooling mechanism that extracts local features from the encoder for query initialization, introducing a temporal inductive bias while maintaining the architectural efficiency of DETR. On the QVHighlights benchmark, Gaussian-DETR provides a scale-specific benefit for long-event retrieval, improving Long mAP by +1.98 points over Moment-DETR with statistical support (p = 0.0111), while the differences in overall recall are statistically inconclusive. Ablations suggest that the Gaussian prior improves scale-aware query initialization by balancing long-event coverage and local temporal specificity. This mechanism provides a parameter-efficient component for studying scale-aware query initialization in DETR-like video moment retrieval models.