Injecting visual semantics into sparse radar: A sensor enhancement framework for robust 3D detection

Abstract Millimeter-wave (mmWave) radar is highly valuable for all-weather autonomous driving perception due to its excellent environmental robustness and range-velocity measuring capability. However, it inherently suffers from severe point cloud sparsity, high noise, and insufficient semantic information, which makes it challenging to distinguish static targets from background clutter. Existing methods heavily rely on multi-sensor fusion or dense geometric distillation, which easily introduce geometric priors that are difficult to support by sparse radar observations. To address these limitations, we propose a novel object-query-driven visual-to-radar semantic distillation method, RCTrans-CD, which utilizes visual object queries as stable semantic anchors. Through query-point contrastive learning, our framework directly associates sparse radar physical returns with high-level target semantics, and incorporates a motion-compensated temporal memory constraint to enhance cross-frame consistency. Extensive evaluation on the nuScenes data set demonstrates that the proposed method achieves a competitive 23.2% mAP for radar-only 3D detection, while significantly improving static object recognition and velocity estimation.