Generative Testing Scenario Image Synthesis for Virtual-Reality Fusion-Based Autonomous Vehicle Testing

With the advancement of autonomous driving technology, a large number of commercial demonstration applications have gradually been carried out worldwide. However, due to the complexity and dynamics of the real traffic environment, some traffic accidents have occurred in recent years. The vehicle-road-cloud system, by deeply integrating technologies such as virtual scenes, real driving environments and vehicle-road coordination, provides an effective solution for improving the safety and reliability of autonomous vehicles on actual roads, which has become a current research hotspot. However, the current virtual-real fusion (VRF) test method mainly targets planning control module. Due to the significant difference between the perception data in the virtual environment and the real environment, it is difficult to test the perception module, thus making it impossible to achieve system-level testing of autonomous driving vehicles. To address this issue, this paper explores a perceptual image generation method that takes the real perceptual images as input and combines them with the background vehicle status in the virtual scene. Based on the latent diffusion model, this paper adopts a generation control module with low-rank decomposition parameters to guide the style of generating images, introduces a trainable encoded copy module to prevent the key data in the images from being damaged, and avoids generating additional uncontrollable details through the latent prompt word fine-tuning module. Finally, a large number of datasets were collected using the CARLA simulation simulator for training and verification. The test results show that the images generated in this paper have been improved under low light conditions at dusk and night.