Synthetic Data in Data-driven Systems

Dataset generation is cumbersome yet of great importance for successful training of machine learning models. Collecting real-world data is expensive and sometimes prohibited, considering e.g. safety aspects or legal restrictions. By generating the bulk of training data by synthetic means it is possible to impose arbitrary and extensive scene randomization for increased data diversity. Methods to quantify similarity between datasets on a statistical level are important tools to detect Out-of-Distribution (OOD) data and domain alignment. We have studied how such methods can be used to correlate model prediction accuracy drop when exposed to OOD-data. Domain adaptation can be applied as an additional step to synthetic data, to decrease the gap to real world datasets, however it can introduce inadvertent label-flipping, a sort of semantic inconsistency between synthetic source and domain adapted output. Therefore, we pursuit another way of reducing the domain gap, by generating high-fidelity digital representations of real-world scenes and objects. We do this through the use of Neural Radience Fields and Gaussian Splats. These methods allow us to render objects of interest for a detection problem, with the perfect annotation of synthetically produced data, and a high degree of realism which we show improves detection accuracy compared to traditionally generated visual content.