Differentiable Targetless Radar-Camera Extrinsic Calibration Based on Detection Attributes

The sparsity and uncertainty of millimeter-wave (MMW) radar point clouds make it difficult to extract features for matching with images, leading to a major challenge for targetless radar-camera extrinsic calibration. To address this problem, we propose a differentiable calibration method based on detection attributes, enabling targetless extrinsic calibration between MMW radar and camera. Our approach utilizes a three-branch neural network to extract cross-modality features between point clouds and images. By leveraging the dynamic properties and radar cross-section values of detected objects, the sparse MMW radar point clouds are augmented, and by calculating the relevance between paired features, the impact of uncertainty in MMW radar detection is resolved. Subsequently, a differentiable probabilistic Perspective-n-Point solver is employed to achieve end-to-end extrinsic parameter estimation without relying on initial extrinsic parameters or specific calibration targets. On the pose dataset constructed from the nuScenes dataset, the proposed method achieved a registration accuracy of 90.10%. Additionally, real-world experiments validate its precision and robustness in radar-camera calibration, demonstrating its effectiveness in targetless scenarios.