Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network

Light Field Angular Super-Resolution (LFASR) addresses the issue where Light Field (LF) images can not simultaneously achieve both high spatial and angular resolution due to the limited resolution of optical sensors. Since Spatial-Angular Correlation (SAC) features are closely related to the structure of LF images, its accurate and complete extraction is crucial for the quality of LF images reconstructed by the LFASR method based on Deep Neural Networks (DNNs). In low-angular resolution LF images, SAC features must be extracted from a limited number of pixels that are at a great distance from each other and exhibit strong correlations. However, existing LFASR methods based on DNNs fail to extract SAC features accurately and completely. Due to the limited receptive field, methods based on regular Convolutional Neural Networks (CNNs) are unable to capture SAC features from distant pixels, leading to incomplete SAC feature extraction. On the other hand, methods based on large convolution kernels and attention mechanisms use an excessive number of pixels to extract SAC features, resulting in insufficient accuracy in extracted SAC features. To solve this problem, we introduce Deformable Convolutional Network (DCN), which adaptively changes the position of limited sampling point using offsets, so as to extract SAC from distant pixels. In addition, in order to make the offset of DCN more accurate and further improve the accuracy of SAC features, we also propose a Multi-Maximum-Offsets Fusion DCN (MMOF-DCN). MMOF-DCN can reduce the exploration range of finding the desired offset, thereby improving the offset finding efficiency. Experiment results show that our proposed method has advantages in real-world dataset and synthetic dataset. The PSNR value in synthetic dataset which have large disparity is improved by 0.45 dB compared to existing methods.