Distilling Knowledge for Designing Computational Imaging Systems

Designing the physical encoder is crucial for accurate image reconstruction in computational imaging (CI) systems. Currently, these systems are designed using an end-to-end (E2E) optimization approach, where the encoder is represented as a neural network layer and is jointly optimized with the computational decoder. However, the performance of E2E optimization is significantly reduced by the physical constraints imposed on the encoder, such as binarization, light throughput, and the compression ratio. Additionally, since the E2E learns the parameters of the encoder by backpropagating the reconstruction error, it does not promote optimal intermediate outputs and suffers from gradient vanishing. To address these limitations, we reinterpret the concept of knowledge distillation (KD)-traditionally used to train smaller neural networks by transferring knowledge from a larger pretrained model-for designing a physically constrained CI system by transferring the knowledge of a pretrained, less-constrained CI system. Our approach involves three steps: First, given the original CI system (student), a teacher system is created by relaxing the constraints on the student's encoder. Second, the teacher is optimized to solve a less-constrained version of the student's problem. Third, the teacher guides the training of the highly constrained student through two proposed knowledge transfer functions, targeting both the encoder and the decoder feature space. The proposed method can be employed to any imaging modality since the relaxation scheme and the loss functions can be adapted according to the physical acquisition and the employed decoder. This approach was validated on three representative CI modalities: magnetic resonance, single-pixel, and compressive spectral imaging. Simulations show that a teacher system with an encoder that has a structure similar to that of the student encoder provides effective guidance. Our approach achieves significantly improved reconstruction performance and encoder design, outperforming both E2E optimization and traditional non-data-driven encoder designs.