Camera-Based Sensor Model : Bridging the Domain Gap between Simulated and Real Images

The use of synthetic data has become increasingly more important for training and evaluating machine learning models, particularly in scenarios and areas where it is hard or costly to obtain real-world data. However, using synthetic data comes with its problems, most notably the domain gap, which can reduce the performance of models trained on synthetic data. This thesis investigates the simulation of a realistic camera sensor model and image processing pipeline to reduce the domain gap between real and synthetically produced images. The pipeline incorporates steps such as color filter array simulation, demosaicing, white balance correction, local contrast enhancement, tone mapping, gamma correction, and final post-processing. An ablation study was performed to evaluate the contribution of each processing step to the perceived realism of synthetic images by measuring color accuracy and perceptual similarity. Furthermore, the impact of the processing steps on object detection performance was assessed using a pre-trained YOLO model. The results show that gamma correction and accurate white balance estimation play a big role in reducing the domain gap, while the inclusion of a color filter array and demosaicing provided no clear benefit at the tested resolution. The study also highlights the difficulties of using auto white balance methods and that incorrect white balance estimation negatively affects both image realism and detection performance.