Light-field generation for 3D light-field display with IARF and adaptive ray sampling

High-quality light-field generation of real scenes based on view synthesis remains a significant challenge in three-dimensional (3D) light-field displays. Recent advances in neural radiance fields have greatly enhanced light-field generation. However, challenges persist in synthesizing high-quality cylindrical viewpoints within a short time. To handle these issues, the instant adaptive radiance field (IARF) method is proposed to enhance the synthesized light-field quality from a set of captured images. In the ray marching process, the adaptive ray sampling technique is presented for resampling within both discrete occupied grids and continuous unoccupied spaces, which ensures that more representative points are acquired, thereby improving image quality. Furthermore, the volumetric sampling consistency (VSC) loss is used for adaptive ray sampling, which maintains the consistency, contributing to shorter training times with high quality. The iterative network structure of IARF is designed to achieve the resampling of points along emitted rays, which ensures the convergence of the density distributions and enhances synthesis accuracy. The distortion loss is introduced to optimize the unbounded scene geometry, and more realistic rendering is achieved. Finally, the expected viewpoint rendering with a backward ray tracing technique is presented to directly render synthetic images based on off-axis light-field image coding. Experimental results validate the effectiveness of our method. The IARF can achieve more detailed viewpoint generation for 360-degree scenes in a shorter training time. The ablation study demonstrates the validity of the proposed VSC loss and utilized distortion loss. Compared to other established methods, an average improvement of 2.14 dB in PSNR is achieved with approximately 9 minutes of training. The IARF can generate synthetic images at arbitrary positions and viewing angles within the scene, rather than being limited to a narrow field of view. Notably, a smooth motion parallax is obtained based on the 3D light-field display with 7680×4320 resolution over a large viewing angle. We believe that the IARF method can facilitate light-field generation by synthesizing real-world scenes, which represent a promising application for 3D light-field display.