Mask-based adaptive response distillation for efficient image super-resolution

Knowledge distillation (KD) is a promising strategy for lightweight image super-resolution (ISR). However, most existing methods rely on vanilla response distillation, which fails to account for the importance of high-frequency structures and treats all pixels equally—often leading to suboptimal learning. To address these issues, we propose a simple yet effective KD framework that is both cost-efficient and architecture-agnostic. Our method introduces: (1) a mask-based adaptive response distillation strategy that emphasizes hard instances and pixels via instance-wise suppression and pixel-wise weighting; and (2) a GT-pretrained student initialization scheme that improves optimization stability and performance. Unlike prior works that add significant training overhead, our framework enhances KD effectiveness without requiring additional modules or labels. Extensive experiments on both CNN and Transformer-based SR models demonstrate that our method consistently outperforms baseline and state-of-the-art KD techniques across multiple datasets and upscaling factors, offering strong generalization and plug-in flexibility. • We identify two key reasons why L1 loss–based response distillation is ineffective in ISR and propose a mask-based adaptive response distillation method to address these issues. • We experimentally demonstrate the importance of student model initialization and show that using a GT-pretrained student significantly improves performance during knowledge distillation. • The proposed method is architecture-agnostic, incurs no additional computational cost, and can be easily combined with other SOTA KD methods to achieve synergistic improvements.