Surface-Continuous Scene Representation for Light Field Depth Estimation via Planarity Prior

Light field (LF) imaging captures both spatial and angular information of the real world, enabling precise depth estimation. However, images are merely discrete expressions of scenes. Limited by imaging technology, LF camera cannot capture the infinite rays emitted by scenes, leading to the discrete information storage (e.g. pixel). Consequently, previous deep learning methods have encountered challenges in accurately extracting depth information from LF images. In this paper, we investigate a surface-continuous scene representation using planarity prior and design PlaneNet, a Plane-based Network that successfully generates highly detailed depth maps for real scenes. Specifically, inspired by the plane assumption that real-world scenes generally yield piecewise smooth surfaces, we refine it to the pixel level for continuous surface approximation, which can overcome the limitations of discrete representation. Rather than explicitly parameterizing planes as multiple coefficients, we propose a novel plane regular sampling operator (PRSO), enabling the network to fit smooth depth surfaces easily. To explore the role of our theory at the feature level, we also introduce PRSO into the intermediate layers of PlaneNet. Experiments show that our method achieves state-of-the-art performance on both synthetic and real-world LF scenes, ranking 1st (MSE) on the HCI 4D Light Field benchmark. Furthermore, we explore the utilization of our representation in multiple LF depth estimation networks, and experiments demonstrate improved performance when surface-continuous representation is applied. Code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/crs904620522/PlaneNet</uri>.