Synthetic Data-Driven Augmentation for Precise 6-DoF Pose Estimation of Building Components in Automated Facility Inspections

This paper tackles the challenge of automating facility inspections by detecting building components, estimating their six-degree-of-freedom (6-DoF) poses (position and orientation), and comparing these estimations to Building Information Modeling (BIM) ground truth data. Vision based Deep learning methods offer promising results in pose estimation. They rely heavily on large annotated image datasets for training, which are often lacking or too specific for building settings. Manually annotating large datasets is both labor-intensive and time-consuming, particularly when they contain hundreds of images. This challenge further underscores the need for efficient and automated methods for dataset annotation, particularly in building environments where dataset size and complexity are significant. We present a study examining the point where the performance of a 6-DoF object detection algorithm is efficient. In particular, we analyze the proportion of real data required for detection and pose estimation models to perform well on double-leaf doors. Synthetic data are generated using the Digital Twin (DT), which is intended to serve as the future reference for BIM in facility inspections. Adding synthetic data reduces rotation error by up to 1.7°, starting from 1,000 images. We analyze the effectiveness of different proportions of real and synthetic data to provide insight into optimizing dataset composition. The accuracy of the 3D center point and the accuracy at 25 cm/25 degrees are better than 90% accuracy when more than 50 real images combined with synthetic data.