A Challenge-Response based Authentication Approach for Multimodal Biometric System using Deep Learning Techniques

Multimodal Biometric System (MBS) is an advanced progression of conventional biometric authentication system, which employ multiple biometric traits to enhance security. However, despite their advantages, these systems are vulnerable to presentation attacks, where adversaries use photos, replay videos or voice recordings to deceive the authentication process. Therefore, this paper proposes a challenge-response based approach using texture-based facial features and multidomain speech features. The challenge-response approach requires the user to utter a random word. Next, the system detects the user’s facial features (eye and mouth motion) and recognized speech text to confirm whether the authentication request originates from a legitimate user or an imposter. The feature-level fusion via concatenation method is used to combine these image-audio features, to reduce the overlap within the feature spaces and data dimensionality. The fused feature vector is then fed into the deep learning driven ensemble classifier CNN-BiLSTM to train and test the fused samples for user authentication. The performance evaluation is carried out using a self-built database with 55 users, achieving 96.81% accuracy, 98.20% precision and an Equal Error Rate (EER) of 3.37%. Moreover, the proposed approach surpasses different cutting-edge MBS, deep learning classifiers and image-audio fusion techniques on various performance metrics. Thus, the results underscore the effectiveness of the deep learning-based MBS in ensuring user authentication and spoof detection, demonstrating its considerable potential in bolstering the security of biometric systems against intricate presentation attacks.