DP-FedFace: Privacy-Preserving Facial Recognition in Real Federated Scenarios

Advanced deep learning-based face recognition models require extensive datasets for optimal performance. However, increasing privacy concerns drive the limitation of face image access on devices to prevent personal information leaks. To address this, federated learning, which allows decentralized data collaboration, has gained popularity. However, traditional federated learning methods risk privacy by transmitting identity proxies to servers. We propose DP-FedFace, a privacy framework specifically designed for a realistic scenario where each client contains only the owner's face images (one identity per client). It uses the difference between human and model perception to eliminate visualization-critical low-frequency components, thus protecting user privacy. We also introduce a novel, learnable privacy cost allocation mechanism that optimizes allocation strategies and adds noise to frequency domain features. Extensive experiments demonstrate that DP-FedFace maintains high recognition accuracy while offers robust privacy protection.