Light field stitching using light field feature points

A single Light field (LF) imaging system tends to have a relatively small field of view (FoV). LF stitching can expand the imaging range and obtain more scene information. This is especially valuable for computer vision applications that require global environmental perception. The existing methods of using 2D feature points to perform sub-aperture image (SAI) matching to achieve LF stitching destroy the spatial and angular consistency of the stitched LF because it does not consider the connection between views. In this paper, a novel LF stitching method is proposed, the innovation of this method is to use LF feature points to replace the 2D feature points. The LF feature points we choose are the Harris feature points detected in the scale-disparity space established using fourier disparity layers (FDL) with circular gradient histogram descriptor. The selected LF feature points contain the coordinate information of feature points on the central SAI, scale information, and disparity information. Based on the disparity values and feature points on the central SAI, the feature points on each SAI can be obtained through disparity transformation and color difference threshold. For each pair of SAIs, the stitching method based on line-guided warping and line-point constraint is then applied to ensure correct alignment of the image content. Experimental results show that using LF feature points can effectively stitch 4D LFs, and preserve the spatial and angular consistency of the stitched LF compared with directly using the image stitching corresponding SAIs based on 2D feature points.