Teacher-student learning of generative adversarial network-guided diffractive neural networks for visual tracking and imaging

Efficiently tracking and imaging interested moving targets is crucial across various applications, from autonomous systems to surveillance. However, persistent challenges remain in various fields, including environmental intricacies, limitations in perceptual technologies, and privacy considerations. We present a teacher-student learning model, the generative adversarial network (GAN)-guided diffractive neural network (DNN), which performs visual tracking and imaging of the interested moving target. The GAN, as a teacher model, empowers efficient acquisition of the skill to differentiate the specific target of interest in the domains of visual tracking and imaging. The DNN-based student model learns to master the skill to differentiate the interested target from the GAN. The process of obtaining a GAN-guided DNN starts with capturing moving objects effectively using an event camera with high temporal resolution and low latency. Then, the generative power of GAN is utilized to generate data with position-tracking capability for the interested moving target, subsequently serving as labels to the training of the DNN. The DNN learns to image the target during training while retaining the target’s positional information. Our experimental demonstration highlights the efficacy of the GAN-guided DNN in visual tracking and imaging of the interested moving target. We expect the GAN-guided DNN can significantly enhance autonomous systems and surveillance.