Hint-DETR: A Transfer Learning Model Based on DETR for Few-Shot Defect Detection

Defect detection, which ensures production stability and final product quality, is a critical part of modern intelligent manufacturing processes. To address the challenge of the difficulty of training the large model networks due to the scarcity of labeled negative samples and detecting defects with ambiguous semantic features, we propose a novel few-shot defect detection method based on transfer learning, called Hint-DETR. Two modules, Memory Bank of Defect and Background (MBDB) and Spatial Feature Information Prompt (SFIP), were added to the DETR network to adapt it to the few-shot defect detection task. The MBDB module can assist the model in learning a memory bank that contains all the defect class feature information to improve the defect perception ability of the model using a small number of defect samples and contains the background suppression loss function designed to reduce the interference of the complex background. The SFIP module extracts spatial feature information from multi-scale feature maps to enhance the DETR network’s detection ability for multi-scale targets, particularly small-scale targets. To assess our method’s performance, we created two few-shot defect datasets using publicly available defect datasets and compared the performance of some state-of-the-art few-shot object detection methods with Hint-DETR. The results of the experiment demonstrate that our method has superior defect detection capability in few-shot settings. Code is available at https://github.com/yunyou727/Hint-DETR.