Adversarially Robust Multimodal Biometric Authentication Using a Hybrid Vision Transformer-DenseNet Face–Ear Fusion Network
This study proposes the Adversarially Robust Hybrid Vision Transformer-DenseNet Fusion (AR-HViTDF) network for sequential face–ear biometric authentication. Despite the increase in security through sequential authentication of modalities, deep architectures are still prone to adversarial perturbations that can change the authentication result through subtle perturbations. The lightweight DenseNet branch learns the finer local texture features, while a compact Vision Transformer (ViT) branch learns the long-distance global structural information. Subsequently, a multi-head cross-attention fusion module is used to learn the interaction between the two branches such that each modality learns from the other. A reinforcement learning (RL)-based agent is employed to adapt the decision threshold exclusively at the decision layer level. The entire system is robustified with projected gradient descent (PGD) adversarial training in addition to a TRADES-style robustness regularizer. Using 5-fold subject-disjoint cross-validation, our approach obtains a clean accuracy of 98.12 ± 0.41%, along with an equal error rate (EER) of 0.71%. Under a PGD-20 attack with ϵ=4/255, the proposed model retains an accuracy of 90.1%, compared with the standard DenseNet baseline. The sequential authentication reduces the false acceptance rate (FAR) by almost 27% compared with the concatenation approach. Cross-dataset experiments further assess the transferability of the learned representations across datasets and acquisition conditions. Differences among the cumulative ablation configurations are assessed using repeated-measures analysis of variance.
Authors
- Abdul Rahaman Wahab Sait (ORCID: https://orcid.org/0000-0001-5445-7899)
- Sanoj Kumar (ORCID: https://orcid.org/0000-0002-0476-4783)
- Musrrat Ali (ORCID: https://orcid.org/0000-0002-4837-1850)
- Swati Rastogi (ORCID: https://orcid.org/0000-0002-2084-2956)
Institutions
- King Faisal University (SA)
- University of Petroleum and Energy Studies (IN)
Publication Details
- Journal
- Mathematics
- Published
- 2026-11-18
- DOI
- https://doi.org/10.3390/math14224239
- Primary Topic
- Biometric Identification and Security
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- King Faisal University