Event-Aware Distilled DETR for Object Detection in an Automotive Context

Autonomous driving systems require robust object detection in complex environments. Event cameras outperform RGB cameras under challenging lighting conditions, but face limitations due to the scarcity of available datasets and lack of specialized training. To narrow the gap between RGB- and event-based detection accuracy and avoid the high complexity of real-time RGB-event fusion, in this paper, we propose a knowledge distillation framework. Our approach uses both modalities during training but relies solely on sparse event data at inference and transfers knowledge from a robust RGB-based teacher model. We build on the success of DETR (DEtection TRansformer) and we leverage an event-aware masked knowledge distillation mechanism, to boost event-based detection accuracy. Experiments on the DSEC-DET dataset demonstrate that our method not only excels in challenging driving scenarios where RGB images are unreliable, but also surpasses the state-of-the-art in event-based object detection.