DefectGen: Few-Shot Defect Image Generation using Stable Diffusion for Steel Surface Analysis

Detecting defects on steel surfaces is crucial for maintaining production standards and minimizing material waste. However, the scarcity of defect images poses a significant challenge in developing robust detection models, which hampers effective defect inspection. To address this issue, we propose DefectGen, a novel data augmentation approach that leverages Stable Diffusion and blending techniques to enhance defect datasets and improve detection performance. With only a limited number of defect images, our method aims to generate a larger and more realistic defect image dataset. The defect image generation process is structured into two stages: the Defect-Free Stage and the Defect Generation Stage. In the Defect-Free Stage, Stable Diffusion is trained on a defect-free dataset to generate realistic steel surface images. The Defect Generation Stage is divided into two phases: defect extraction and modification, followed by blending. In the first phase, defects are extracted from a small number of defect images and modified according to specified conditions. The second phase involves deep blending of the modified defects into the generated defect-free images based on the provided locations, resulting in realistic defect images. Experiments on our steel surface dataset demonstrate the effectiveness of DefectGen, leading to a 11.91% improvement in Mean Average Precision (mAP) for defect detection.