Cerberus: Accurate Real-Time Object Detection System Under Adverse Weather Conditions via Multimodal Fusion

Multi-modal data-driven object detection typically depends on large-scale datasets. However, under adverse weather conditions, these datasets often exhibit a long-tail distribution, resulting in a substantial decline in detection performance and posing risks to system safety in applications such as autonomous driving. To this end, we propose Cerberus, a real-time and high-accuracy object detection system, which consists of two key components: (i) a<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> similarity-driven proposal extraction</i> component, which utilizes pre-trained detectors and convolutional neural networks to enhance low-quality features and suppress noise across image, LiDAR, and radar data, followed by aiming to extract and refine multi-modal features, thereby improving the quality of object proposals; (ii) an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">adaptive weight-based box enhancement</i> component, which integrates a frozen-weight regionbased convolutional network sub-network with a sensor fusion mechanism, followed by refining bounding boxes by normalizing confidence vectors and dynamically weighting multi-modal features, ensuring accurate detection under adverse weather conditions. Experimental results demonstrate that Cerberus surpasses existing baselines, yielding performance gains of 2.7% in the mean average precision and 2.0% in the nuScenes detection score under adverse weather conditions.