Using Synthetic Images to Improve and Test Object Detection in the Context of the Autonomous Vehicle

In the context of the autonomous vehicle, to validate and aspire to attain a certification, one would have to test the embedded AI, from which result its predictive planning and action, on every situations the vehicle may encounter, especially the rare and critical situations known as Edge and Corner Cases. However, obtaining pertinent data proves challenging due to the high costs associated with image collection and annotation, as well as the inherent dangers of certain situations. One proposed solution is the generation of synthetic data through simulation software. Using a rendering game engine through a collaboration with Unity Technologies<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>https://unity.com/, we built a simulator capable of generating images of multiple edge and corner scenarios in various environmental contexts (luminosity, weather and windshield condition). In this study, we conduct multiple experiments using the YOLOv7 [1] object detection model to demonstrate the utility of synthetic data. After generating a full synthetic dataset, we first exhibit the domain gap between synthetic and real images, then we demonstrate that despite this domain gap, synthetic data can be used to identify gaps in knowledge and potential flaws in the model, as well as to refine and enhance its performances (up to a 2% increase in [email protected]).