Pose Guided Portrait View Interpolation from Dual Cameras with a Long Baseline

We introduce a novel method for interpolating views between two fixed cameras to create a free-viewpoint experience comparable to advanced telepresence systems. Inspired by Video Frame Interpolation (VFI) works, our proposed method spatially interpolates between camera views, treating our setup as a unique frame interpolation scenario. Due to the occlusion that exists in areas like the face contour from two cameras with a long baseline, which results in serious artifacts with previous VFI methods. To mitigate this issue, we integrate face pose information of pitch, yaw and roll angles in our method as a robust guidance prior to improve pixel mapping in occluded regions. Furthermore, we synthesize a large portrait-focused multi-view dataset facilitating training of our proposed model. Our multi-stage flow refinement model progressively refines the quality of the bi-directional flows which eventually results in improved interpolated view details. With the flexible setup and fast inference speed of our proposed method, a practical and cost-effective solution can be implemented for telepresence systems without the need for expensive hardware or extensive computational resources.