Transient video interpolation for dynamic non-line-of-sight imaging

Non-line-of-sight (NLOS) imaging has seen rapid progress recently. However, owing to the limited sensor capacity, dynamic NLOS imaging still encounters challenges in dynamic scene analysis. Existing approaches reduce the number of sampling points or shorten the unit acquisition time to increase the frame rate, yet at the cost of lowered spatial resolution or signal-to-noise ratio of the measurements. To simultaneously achieve high-resolution and high-frame-rate transient measurements, we introduce <b>TransVID</b>, the first <b>Trans</b>ient <b>V</b>ideo <b>I</b>nterpolation method built upon the <b>D</b>iffusion model. Given consecutive low-resolution measurements at a low frame rate, TransVID maps the upsampled measurements into the latent feature space via an encoder with tailored spatial-temporal attention. Subsequently, a conditional diffusion process is performed in the latent space to interpolate the latent features of the intermediate frames, which are then decoded and reconstructed to the high-resolution transient frames. Experimental results demonstrate that TransVID breaks the current frame rate limit of physical capture in NLOS imaging, successfully recovers hidden objects with a 128 × 128 resolution at 16 FPS from an original video of a 16 × 16 resolution at 4 FPS.