Reliable and Balanced Transfer Learning for Generalized Multimodal Face Anti-Spoofing
Face Anti-Spoofing (FAS) is essential for securing face recognition systems against presentation attacks. Recent advances in sensor technology and multimodal learning have enabled the development of multimodal FAS systems. However, existing methods often struggle to generalize to unseen attacks and diverse environments due to two key challenges: (1) Modality unreliability, where sensors such as depth and infrared suffer from severe domain shifts, impairing the reliability of cross-modal fusion; and (2) Modality imbalance, where over-reliance on a dominant modality weakens the model's robustness against attacks that affect other modalities. To overcome these issues, we propose MMDG++, a multimodal domain-generalized FAS framework built upon the vision-language model CLIP. In MMDG++, we design the Uncertainty-Guided Cross-Adapter++ (U-Adapter++) to filter out unreliable regions within each modality, enabling more reliable multimodal interactions. Additionally, we introduce Rebalanced Modality Gradient Modulation (ReGrad) for adaptive gradient modulation to balance modality convergence. To further enhance generalization, propose Asymmetric Domain Prompts (ADPs) that leverage CLIP's language priors to learn generalized decision boundaries across modalities. We also develop a novel multimodal FAS benchmark to evaluate generalizability under various deployment conditions. Extensive experiments across this benchmark show our method outperforms state-of-the-art FAS methods, demonstrating superior generalization capability.
Authors
- Jun Wan (ORCID: https://orcid.org/0000-0002-4735-2885)
- Shuai Wang (ORCID: https://orcid.org/0000-0001-7253-785X)
- Rizhao Cai (ORCID: https://orcid.org/0000-0002-7114-8462)
- Xiaochun Cao (ORCID: https://orcid.org/0000-0001-7141-708X)
- Alex C. Kot (ORCID: https://orcid.org/0000-0001-6262-8125)
- Xun Lin (ORCID: https://orcid.org/0000-0001-8387-4245)
- Ajian Liu (ORCID: https://orcid.org/0000-0002-7788-9368)
- Zitong Yu (ORCID: https://orcid.org/0000-0003-0422-6616)
- Zhen Lei (ORCID: https://orcid.org/0000-0002-0791-189X)
- Yi Yu (ORCID: https://orcid.org/0000-0003-2730-9553)
Institutions
- Sun Yat-sen University (CN)
- Nanyang Technological University (SG)
- Chinese Academy of Sciences (CN)
- Institute of Automation (CN)
- Great Bay University
- Beihang University (CN)
Publication Details
- Journal
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2025-05-26
- DOI
- https://doi.org/10.1109/tpami.2025.3573785
- Citations
- 7
- Primary Topic
- Biometric Identification and Security
- Type
- article
- Field-Weighted Citation Impact
- 10.40