DiFace: Cross-Modal Face Recognition through Controlled Diffusion

Diffusion probabilistic models (DPMs) have exhibited exceptional proficiency in generating visual media of outstanding quality and realism. Nonetheless, their potential in non-generative domains, such as face recognition (FR), has yet to be thoroughly investigated. Meanwhile, despite the extensive development of multi-modal FR methods, their emphasis has predominantly centered on visual modalities. In this context, FR through textual description presents a unique and promising solution that not only transcends the limitations from application scenarios but also expands the potential for research in the field of cross-modal FR. It is regrettable that this avenue remains underutilized, a consequence from the challenges mainly associated with three aspects: 1) the intrinsic imprecision of verbal descriptions; 2) the significant gaps between texts and images; and 3) the immense hurdle posed by insufficient databases. To tackle this problem, we present DiFace, an end-to-end solution that effectively achieves FR via text through a controllable diffusion process, by establishing its theoretical connection with probability transport. Our approach not only unleashes the potential of DPMs across a wide range of tasks but also showcases a remarkable improvement in accuracy for text-image FR, as evidenced by our experiments on verification and identification.