Motion deblurring of long exposure images in low light condition by selective fusion of visible and near infrared images

Shooting in low light condition needs prolonged exposure time to take a high quality image by increasing the amount of incident light into a camera sensor. However, long exposure images contain motion blur by object movement. Since near infrared (NIR) images are more robust to low light condition without color information than visible (VIS) images, multi-sensor fusion of VIS and NIR images provides a viable solution to low light imaging for high quality photographs. In this paper, we propose motion deblurring of long exposure images in low light condition by selective fusion of VIS and NIR images. To extract features at different depths, we combine attention with edge feature distillation in the fusion network. We obtain foreground and background features from VIS and NIR images using a segmentation mask. In the training phase, we use the segmentation mask in the loss function to preserve foreground objects while removing motion blur and shadows. For training, we generate a dataset of VIS and NIR images in low light condition using JAI AD-130 GE camera. We manually generate the segmentation mask using Photoshop function and add motion blur into the foreground. Experimental results show that the proposed fusion network successfully removes motion blur and shadows while reproducing realistic color in the fusion results as well as outperforms competing methods on several information-based metrics. The proposed method achieves the highest mutual information (MI) and feature mutual information (FMI), indicating its superior ability to preserve complementary information from the source images while maintaining important structural features. The code is available at https://github.com/Xidian-Media-Lab/motion-deblur .