Automatic Multi-Sensor Dataset Generation in Autonomous Vehicle Environments

This paper presents a comprehensive method for dataset construction, utilizing 3D object detection to automatically label objects detected by LiDAR sensors and synchronizing multi-sensor labeling through coordinate calibration, thereby automatically generating image and radar datasets that support various learning algorithms. Initially, the cocalibration from the camera, radar, and LiDAR sensors is conducted to standardize the coordinate system based on the LiDAR. The camera output includes image information, encompassing object depth and related data. The radar sensor, particularly in automotive applications, returns data on the position of objects in front of the vehicle. Further, the Hungarian Algorithm is employed to analyze the association between radar and camera-detected objects. The proposed collaboration process with software workflow for automatic dataset generation with multi-sensors is detailed in this study. Finally, the preliminary results from sensor fusion over single-sensor modalities to object detection applications are presented to facilitate the efficient and rapid development of our approach to multi-sensor dataset generation, which is still extremely limited to the optical counterparts in autonomous vehicle environments.