Advances in object detection for autonomous driving using mmwave radar and camera: A comprehensive survey

Propelled by advancements in deep learning technology, autonomous driving has undergone rapid development in recent years. Object sensing, as the core technology of autonomous vehicles, is undergoing a transformation from traditional single-modal sensing to multi-modal sensing, aiming to perceive environmental information in a robust, accurate, and real-time manner to adapt to increasingly complex application scenes. This paper aims to offer a comprehensive guide for radar-camera fusion, focusing on the object detection task. Firstly, detailed explanations are provided regarding the characteristics, principles, data processing methods, and representations of radar and camera data. Secondly, we analyze the evaluation metrics of object detection via a group-independent form and summarize existing radar-camera fusion datasets. Thirdly, we comprehensively review and analyze key issues in radar-camera fusion across five hierarchies (why, what, where, when, and how to fuse). Fourthly, we discuss and anticipate potential challenges and emerging research directions from the view of the robustness, accuracy, and real-time performance for object detection.