3D Depth Experience in VR from Monocular Video

Virtual Reality (VR) lives from the experience of diving into a world which is perceived as a three-dimensional environment. Regular two-dimensional video materials captured in the real world are only partially suitable to convey the real feeling of presence in VR. This paper describes the design and implementation of a 3D effect on a video in VR through the use of only a single monocular video as input. This is realized through a monocular depth estimation algorithm, which pre-processes the video and extracts depth information, creating a corresponding depth map video. An application running on a VR headset uses this depth map to create the 3D effect through projecting the video onto multiple spheres at different sizes and distances from the viewer, depending on the corresponding depth value. We describe the entire process and show a few examples, analyzing the individual algorithmic steps.