Knowledge Distillation Using Global Fusion and Feature Restoration for Industrial Defect Detectors

Deep learning technology has been widely applied in industrial quality inspection tasks to improve the accuracy of defect detection, object recognition, and classification. However, in the task of object detection, both feature-based and regression-based traditional knowledge distillation methods impose overly strict constraints on the student model. To address the above issues, this article proposes Global Fusion and Feature Restoration Knowledge Distillation(FRD). FRD integrates contextual information into the channel through a Global Fusion Module (GFM), and further utilizes attention mechanisms to adaptively focus on the distillation region after separating the foreground background. FRD also uses a Mask Feature Restoration (MFR) to mask and restore a portion of student features, improving the learnability of the model. At the same time, FRD adopts a comprehensive approach of multiple loss superposition constraints, rather than simply using MSE losses to imitate the features of teachers. Experiments have shown that FRD can effectively improve model performance in industrial defect detection tasks. On the aluminum surface defect dataset, FRD increased the mAP index of RetinaNet-Res50 from 57.7% to 61.7%. We also confirmed the effectiveness of FRD for general object detection on the Coco dataset. On a randomly selected COCO dataset containing 4000 images, FRD increased the mAP metric of RetinaNet-Res50 from 21.5% to 43.6%.