A Comparative Analysis of Multi-Modal Semantic Perception Tasks and Datasets for Mobile Robotics

Data is key for semantic perception tasks, such as object detection and semantic segmentation. This is particularly true for multi-modal perception, where different sensors such as LiDAR and camera are fused to improve overall performance. While many public datasets for mobile robotics and adjacent domains exist, a comprehensive comparison of these datasets is still lacking. To address this, and to provide the research community with an overview of the available datasets, this paper conducts a large-scale meta-analysis to identify and compare the most relevant datasets for multi-modal perception tasks in mobile robotics. We provide an evaluation of 31 different datasets across six different tasks, and summarize key challenges, opportunities, and future directions associated with datasets.