Sensor-aware Offline Synthetic-to-Real Adaptation for Multimodal Driving Datasets: Shape Preserved, Signal Improved

In the field of autonomous driving, simulators have gained growing attention as they provide a cost-effective environment to collect data with various sensor modalities across diverse driving scenarios. Yet, the domain gap between real and synthetic data remains a significant barrier. To address this, numerous methods have been proposed to enhance the usability of synthetic data by bridging the gap. However, many of those approaches often focus on a single sensor setup, either a monocular camera or a LiDAR, potentially leading to inconsistent domain adaptation across sensors. As recent perception models actively employ sensor-fusion approaches like camera+LiDAR, we propose Sensor-wise Offline Dataset Adaptation (SODA), designed for a multimodal synthetic driving dataset with multi-view cameras and a LiDAR. It ensures consistency between sensor modalities after domain adaptation by enhancing signal information (camera pixel and LiDAR intensity values) while preserving their shape (image contents, point cloud geometry). To validate our method, MORDA (synthetic) and nuScenes (real) are opted as source and target datasets. In our extensive evaluation, incorporating SODA-applied MORDA during training of BEVFusion (camera+LiDAR) achieves mAP gain of 1.5% compared to training the same model solely on nuScenes. Further, ~2% improvement in mAP is observed under challenging adversecondition scenarios.