Learning Scene Semantics From Vehicle-Centric Data for City-Scale Digital Twins

The creation of digital twins of cityscapes requires the understanding the semantics of relevant objects encountered in the scene, with classes possibly not well covered in standard datasets. In this paper, we assess the generalization capabilities of models for semantic segmentation of street scenes trained on synthetic data to real images. In order to perform assessment on real data, we create a dataset from images obtained from a vehicle-based capture setup, and propose a highly automated annotation pipeline for generating ground truth. We train three state of the art segmentation models on our own synthetic dataset, and evaluate the methods on real data.