Diffdefect: A Diffusion Model-Based Framework for Simulating Defects in Industrial Components

Recent advances in deep learning have revolutionized the field of surface defect detection in industrial vision systems, especially within the realm of industrial equipment maintenance, where timely and accurate inspections play a crucial role in preventing critical failures and ensuring smooth operations. Nevertheless, two major challenges still remain: (1) the high cost associated with acquiring pixel-level annotated defect datasets in real-world industrial settings and (2) the severe class imbalance that tends to favor minor defects over safetycritical anomalies. To address these issues, in this work, we introduce DiffDefect, a novel approach that combines diffusion models with DreamBooth and LoRA fine-tuning, aiming to enhance both performance and computational efficiency. Specifically, our method greatly enriches the existing defect dataset, which consists of five categories, by generating <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{4, 0 0 0}$</tex> highly realistic samples based on the original defect data. Experimental results show that DiffDefect is capable of producing diverse and realistic defect images that not only consistently match the surrounding textures but are also adaptable to different materials and defect types. Moreover, when integrated with existing state-of-the-art classification algorithms, our approach increases accuracy from <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{9 5. 8 9 \%}$</tex> to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{9 9. 6 7 \%}$</tex>, thereby demonstrating its great potential in practical industrial applications.