Bridging the Reality Gap: A Framework for Synthetic Data Generation in Smart Manufacturing

Automated material inspection in smart manufacturing relies on extensive, diverse, and annotated datasets for robust deep neural networks (DNNs). However, acquiring real industrial data, especially with varied defects like occlusion and misalignment, is challenging and costly. This study presents Bridging the Reality Gap (BRG), a novel synthetic data generation framework. The BRG pipeline uses a high-fidelity 3D CAD model to simulate camera perspectives and physically based rendering with HDRI to generate diverse load station samples under various conditions. It systematically creates normal and abnormal samples, including geometric and photorealistic obstructions for misalignment and occlusion. Key contributions include developing the BRG framework to bridge the reality gap in robotic training and demonstrating that BRG generated synthetic data significantly enhances DNN performance. The framework also optimises annotation efforts through a symmetric data generation approach. After preprocessing and hyperparameter tuning, transfer learning is applied to a YOLOv5 model using these synthetic samples. Results show BRG’s significant performance for load station inspection; the combined synthetic dataset (D3) achieved 100% Precision, 93.83% Recall, and 96.82% F1 on real industrial samples.