A swin transformer enhanced reverse knowledge distillation model for industrial anomaly detection via window-aware stochastic local perturbation

Abstract Industrial anomaly detection is critical for quality control in manufacturing, yet it remains challenging due to the scarcity of defective samples and the diversity of potential anomalies. This paper proposes TRKDM-P, a two-stage reverse knowledge distillation framework enhanced by a Swin Transformer backbone and a window-aware stochastic local perturbation strategy. In the first stage, the student decoder acquires fundamental reconstruction capabilities by mimicking frozen teacher features. In the second stage, a window-aware masking strategy simulates realistic anomalies within Swin Transformer windows, enforcing the model to learn repair mechanisms and improving generalization to unseen defects. To further enhance reconstruction fidelity and localization precision, we introduce an Attention-Guided Feature Enhancement module and a normalized feature repository that anchors the decoder to normal patterns. Extensive experiments on the MVTec AD dataset demonstrate that TRKDM-P achieves state-of-the-art performance, with image-level and pixel-level AUROCs of 99.4% and 98.1%, respectively. Additional evaluations on BTAD and MPDD datasets confirm its strong generalization. However, the use of a Swin-B backbone and auxiliary modules increases model parameters to 88.1M, highlighting a trade-off between accuracy and efficiency. This work underscores the potential of transformer-based knowledge distillation for industrial anomaly detection while identifying avenues for lightweight deployment.