Simulation of LiDAR Under Fog with Generative Adversarial Networks for Robust 3D Object Detection

Abstract Collecting real LiDAR data from actual scenarios in adverse weather is an expensive and time-consuming process, yet crucial for developing robust perception algorithms. Publicly available datasets published for relevant purposes are often limited to acquisitions under clear weather, which leads to a lack of diversity for the most challenging and unexpected situations that can be encountered in the real world. Virtual datasets that are generated to include emulated adverse weather conditions will significantly facilitate the development and testing processes of more robust perception algorithms. In this paper, we propose a Wasserstein Generative Adversarial Network (WGAN)–Long Short-Term Memory (LSTM) network architecture trained on real continuous LiDAR point clouds gathered in a foggy environment. The output of the generator portion of the network is projected onto a fully simulated virtual scenario. Instead of directly addressing the domain gap induced by this method, we show indirectly that training on the generated dataset improves the performance and robustness of 3D object detection algorithms.