Effects of Facial Hair on Face Recognition

A person’s facial hairstyle, such as presence and size of beard, can significantly impact face recognition accuracy. While previous research has examined the facial hair effect using binary attributes, no work utilizes a segmentation model to capture the full extent of the facial hair. To investigate the effect of facial hair size in a rigorous manner, we first created a set of fine-grained facial hair annotations to train a segmentation model. Cross-dataset evaluation is performed and accuracy across African-American and Caucasian face images is reported. We then use our facial hair predictions to categorize image pairs according to the degree of difference or similarity in the facial hairstyle. We find that the False Match Rates for image pairs with different categories of facial hairstyle varies by a factor of over 10 for African-American males and over 25 for Caucasian males on MORPH dataset. Also, False Non-Match Rates of 4 race categories on BA-Test dataset are analyzed to measure the accuracy bias in unconstrained settings. Our findings suggest that, while facial hair can cause a shift in similarity score distributions, this effect can be mitigated by employing an adaptive threshold based on facial hair predictions. Facial hair annotations: https://github.com/kaganozturk/Effects-of-Facial-Hair-on-Face-Recognition.