Front-to-Side Hard and Soft Biometrics for Augmented Zero-Shot Side Face Recognition

Face recognition is a fundamental and versatile technology widely used to identify individuals. The human face is a significant nonintrusive biometric modality, attracting numerous research studies. Still, much less focus has been on side-face views, with the majority merely or mainly concentrating on the frontal face. Despite offering fewer traits than the front viewpoint, the side viewpoint of the face is a crucial aspect of an individual's identity and, in numerous cases, can be the only available information. Our research proposes new soft biometric traits based on the face anthropometric that can be invariantly extracted from the front and side face. We aim to extract and fuse them with vision-based deep features to augment zero-shot side face recognition. Our framework uses the person's front face information solely for training, then uses their side face information as the only query for biometric matching and identification. For performance evaluation and comparison of the proposed approach, several feature-level fusion experiments were conducted on the CMU Multi-PIE dataset. Our results demonstrate that fusing the proposed face soft traits with the ResNet-50 deep features significantly improves performance. Furthermore, adding global soft biometrics to them improves the accuracy by up to 23%.