Learning Cubic Field Representation from A Single Panorama for Virtual Reality

Panoramic images provide comprehensive scene information and are suitable for VR applications. Obtaining corresponding depth maps is essential for achieving immersive and interactive experiences. We propose a novel method that learns a cubic field composed of multiple MPIs from a single panoramic image for depth estimation. Our method employs cubemap projection to transform an ERP image into six faces and extract the MPIs for each. An attention-based blending module is employed to learn correlations among the MPIs, constructing a cubic field representation. Furthermore, a novel sampling strategy is introduced for rendering novel views from the cubic field at both cubic and planar scales. The entire pipeline is trained using photometric loss calculated from rendered views within a self-supervised learning approach. Experiments demonstrate the superior performance of the proposed method and highlight its effectiveness in downstream applications.