Sensor Fusion-Based Classification Fault-Tolerant System for Detected Objects in Autonomous Cars

This paper proposes a fault-tolerant system based on sensor fusion for the classification of detected objects to ensure the safety of autonomous driving. The system addresses and refines misclassifications made by black-box object detectors. Raw sensor data from camera, Light Detection And Ranging (LiDAR), and radar are extracted within the bounding box region identified by the black-box detector. The classification is based on the extracted images and point cloud data, with the inclusion of a ‘fault class’ to detect misclassifications by the black-box detectors. This approach also consolidates class labels to improve classification reliability and reduce computational costs. A dataset is built to evaluate the performance of system. The classification performance is analyzed based on different sensor types (camera, LiDAR, radar), machine learning techniques (attention mechanisms), and sensor fusion methods (early, middle, late). The classification result is obtained by probabilistically fusing the outputs of both black-box and fusion-based classifiers. The results demonstrate that this fusion-based approach significantly improves the reliability of the classification, providing a robust solution to safe autonomous driving.