High-quality video interpolation via event information incorporation

To enrich low-frame-rate video details over time, optical flow-based video interpolation is commonly used. It works by synthesizing multiple transitional frames between existing ones. However, it struggles to reconstruct non-linear and high-speed motion trajectories. To better capture dynamic scenes with low-frame-rate conventional cameras, this work proposes a complementary framework integrating low-frame-rate conventional cameras and event cameras, enhancing reconstruction accuracy of dynamic scenes by leveraging event-driven dynamic cues. The proposed framework 1) uses event streams to separate dynamic foregrounds from static backgrounds, 2) generates transitional frames between keyframes through spatiotemporal event correlations, and 3) combines deformable optical flow with multi-scale compensation. Empirical studies indicate that the proposed method outperforms state-of-the-art methods for interpolation in low-frame-rate videos.