Data-Efficient Domain Transfer for Instance Segmentation for AR Scenes

Augmented Reality (AR) applications rely heavily on the accurate detection and segmentation of objects in order to seamlessly integrate virtual content into real world environments. However, achieving robust object detection and segmentation in AR scenes remains challenging, in particular when specific classes need to be covered. Due to the lack of sufficient real annotated data, training can rely on synthetic data, which has the issue of the domain gap between synthetic and real-world images. This paper addresses the challenge of generalisation in object detection/instance segmentation models trained with synthetic data. It presents techniques to improve the generalisation capability of object detection models trained on synthetic data for real-world applications. Our method is based on the model soup approach, where models trained with different data subsets and hyperparameters are combined to achieve better generalisation performance. The experimental results on the validation dataset of the ADAPT sim2real Object Detection Challenge 2023 demonstrate the effectiveness of our approach in achieving good generalisation performance. This marks a significant step towards more reliable and adaptable object detection and segmentation approaches for AR applications.