Multi-Stage Knowledge Distillation for Progressive Image Deblurring in Teacher-Student Networks

Image deblurring is the process of recovering a high-quality image from a degraded image, which can be lost sharpness by blur filters, noise, compression, or other degradation factors. Image deblurring is a challenging task, as it requires dealing with complex and ill-posed inverse problems, and balancing the trade-off between performance and efficiency. This paper introduces a multi-stage knowledge distillation approach using teacher-student network interactions. Our method employs a three-stage teacher network combining Window-based Transformer and Unet models for the first and second stage, followed by a Channel-level Transformer as the third stage to enhance detail extraction. The student network, streamlined for speed, mirrors the teacher's structure with lower complexity, incorporating a standard Unet model and channel attention mechanism. A new Supervised Attention Module is introduced for effective knowledge transfer and feature enhancement. We evaluate our method on various datasets and show significant enhancements in image deblurring quality, while balancing performance and model efficiency. Our findings suggest that our method has great potential for real-world applications and opens up new possibilities for future research in image deblurring techniques.