Gaussian membership-self-attention encoder fusion network (GMSAEF-Net) for the multimodal biometric recognition

This paper presents a secure and robust multimodal biometric (MB) authentication framework that synergistically combines deep learning (DL) with blockchain technology (BCT) to address the inherent limitations of unimodal systems and challenges such as feature redundancy in multimodal biometric data. The system utilizes vein images from the dorsal hand, palm, and fingers, enhanced using contrast-limited adaptive histogram equalization (CLAHE) to improve local contrast. Discriminative features are extracted via an optimized ResNet50 model, fine-tuned through a mutation-based wild horse optimization (MWHO) algorithm to ensure superior representation. A key innovation is the proposed Gaussian Membership Self-Attention Encoder Fusion Network (GMSAEF-Net), a novel architecture that effectively manages multimodal feature correlation and redundancy. Unlike conventional fusion methods, GMSAEF-Net employs fuzzy Gaussian membership functions to assign modality-aware attention weights and incorporates a self-attention mechanism to model interdependencies across feature channels, thereby enhancing the compactness and discriminability of the fused features. To ensure data integrity and resistance to tampering, the system leverages homomorphic encryption within a blockchain framework to generate unique cryptographic keys per modality. A hybrid loss function combining weighted cosine similarity and cross-entropy further improves classification performance. Experimental results on three benchmark multimodal vein datasets demonstrate high recognition accuracy of 98.77% and a significantly reduced Equal Error Rate (EER) of 9.78%, outperforming existing approaches and validating the proposed system’s effectiveness for secure biometric authentication applications.