Multitask Hybrid Knowledge Distillation for Unsupervised Anomaly Detection

Detecting both logical and structural anomalies in an unsupervised anomaly detection task is a significant challenge due to the inherent differences between the two types of anomalies. The use of two-branch knowledge distillation to deal with these two types of anomalies separately is a generalized approach. However, existing methods often design dual branches separately, which does not effectively utilize the shared information between these two branches. Also, due to the introduction of bottleneck layers, a large amount of detailed information is often lost during the reconstruction process, resulting in many false positives. To overcome these drawbacks, we structure the student network as a multitask model to enhance its feature extraction capability, thereby improving its ability to distinguish between logical and structural anomalies, especially under the constraint of limited training data. In addition, we incorporated a self-supervised distillation loss within the logical detection branch and trained the model using a hybrid distillation approach. By leveraging the differences in features between self-distillations to detect logical anomalies, we effectively minimized the false positives that often arise from image reconstruction blurring due to feature compression in the logical branch. We conducted experiments on three well-known anomaly detection datasets to demonstrate the effectiveness of our approach. In particular, on the challenging MVTec LOCO AD dataset, our method achieved impressive results with a pixel-level sPRO of 82.9% and an image-level area under the receiver operating characteristic curve (AUROC) of 91.0%.