Thermal-Visible and Near Infrared-Visible Heterogeneous Face Recognition in Small Scale Datasets

Thermal or Near Infrared imaging sensors, capable of working in poor lighting conditions are currently being used in surveillance systems to address the issues due to low illumination. However, the face images in the existing image galleries are in the visible spectrum. Hence, there is a need to match the query face images captured in the thermal spectrum with face images in the image gallery. Since these two images are captured in two different spectrums, the thermal/NIR images (τ ) have been converted to visible images (V′). Then it is matched with corresponding ground truth visible images (V ) to accomplish the task of face recognition. In this work, the existing Pix2Pix GAN model has been used to generate visible face images from their corresponding thermal/NIR face images. Further, a light Convolution Neural Network model has been proposed to perform heterogeneous face recognition between (V′) and (V ). The proposed model has given heterogeneous face recognition accuracy of 72.26%, 79.71%, and 87.31% for IRIS-thermal, CARL-Thermal, and CARL-NIR datasets, respectively. Further, the False Acceptance Rate (FAR) score of 0.0095%, 0.0067%, and 0.0025% and False Rejection Rate (FRR) score of 0.3190%, 0.1937%, and 0.0843% have been produced for the IRIS-thermal, CARL-Thermal, and CARL-NIR datasets, respectively. Finally, the remarkable reduction in the number of parameters is a major observation of the proposed method. It has been observed that the total number of trainable parameters is 0.28 M, making the model significantly light thereby reducing the computational cost. As a result, it can be deployed efficiently on edge devices.