Advanced Defect Analysis With Self-Supervised Pretraining and Knowledge Distillation

Defect detection is vital for quality control in industrial production. This paper presents a method leveraging pre-trained contrastive learning models to enhance defect detection. We investigate the SimCLR model with two training strategies: Training from Scratch and Fine-tuning. Fine-tuning involves adjusting pre-trained weights to adapt the model for defect detection, leading to superior performance. Training from Scratch initializes the model parameters anew, allowing for a more customized approach to anomaly detection. Our method incorporates the reverse knowledge distillation model with a SimCLR-trained backbone, leveraging self-supervised and unsupervised learning techniques using only defect-free samples, enhancing practicality. We evaluate using two well-recognized anomaly detection datasets, featuring high-resolution images of various industrial product categories with detailed annotations. Experiment results demonstrate that both training strategies outperform state-of-the-art methods with higher area under the receiver operating characteristic curve (AUROC), highlighting the effectiveness of our method.