FaDE: A Face Segment Driven Identity Anonymization Framework For Fair Face Recognition

Current face recognition (FR) algorithms frequently encounter discrimination issues in terms of various attributes (e.g., gender, age) due to the biased demographic distribution of the training datasets towards specific groups. In this paper, we study an identity protected fair FR problem where the goal is to augment the datasets with external face images while ensuring the anonymity of the corresponding face identities. Our problem is motivated by the limitation of current fairness driven data augmentation approaches that directly utilize the external face images accessed by FR algorithm developers while ignoring the protection on the face identities of the image owners. To address the problem, we develop FaDE, a face segment driven identity anonymization framework that augments biased face image datasets by identifying specific face segments with diversified demographic characteristics from external face images but with least identity disclosure, and then reconstructing the segments to full face images with new identities. As a result, the augmented dataset is under a more balanced demographic distribution and improves the fairness performance of the optimized FR algorithms. We evaluate FaDE on two public face datasets, CelebA and LFW that suffer from various demographic imbalance. The results show that FaDE significantly enhances both fairness and accuracy performance of the optimized FR algorithms, while keeping effective anonymity for the identities of external face images.