Consensus-Based 3D View Generation from Noisy Images

The real-time synthesis of 3D views, facilitated by convolutional neural networks like NeX, is increasingly pivotal in various computer vision applications. These networks are trained using photographs taken from different perspectives during the training phase. However, these images may be susceptible to contamination from noise originating from the vision sensor or the surrounding environment. This research meticulously examines the impact of noise on the resulting image quality of 3D views synthesized by the NeX network. Various noise levels and scenes have been incorporated to substantiate the claim that the presence of noise significantly degrades image quality. Additionally, a new strategy is introduced to improve image quality by calculating consensus among NeX networks trained on images pre-processed with a denoising algorithm. Experimental results confirm the effectiveness of this technique, demonstrating improvements of up to 1.300 dB and 0.032 for Peak Signal Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM), respectively, under certain scenes and noise levels. Notably, the performance gains are especially significant when using synthesized images generated by NeX from noisy inputs in the consensus process.