An Enhanced Siamese Neural Network with Attention-Adam Optimization for Bimodal Biometric Authentication in Mobile Financial Systems

The increasing use of mobile financial services has intensified the need for authentication mechanisms that combine security, usability, and computational efficiency on resource-constrained devices. Passwords and unimodal biometric systems can expose mobile financial applications to credential compromise, presentation attacks, environmental variability, and single-modality failure. This study proposes an enhanced bimodal biometric authentication framework that combines an Enhanced Siamese Neural Network (ESNN) for facial verification with hardware-backed fingerprint authentication. The facial model uses a ResNet-18 backbone enhanced with Squeeze-and-Excitation (SE) blocks for channel-wise feature recalibration. A novel Attention-Adam (Attn-Adam) optimizer is formulated using a sliding window of historical gradients and temporal attention weights to emphasize consistent optimization directions and reduce the effect of noisy gradient updates. Facial data were obtained from the Labeled Faces in the Wild (LFW) dataset, comprising 13,233 images from 5,749 identities. The thesis reports a 70:20:10 training-validation-test split and balanced positive and negative contrastive pairs. The proposed system achieved an ROC-AUC of 0.9892, EER of 4.20%, precision of 95.86%, recall of 95.74%, and F1-score of 95.80%. In the reported ablation study, the baseline ResNet-18 with standard Adam achieved AUC = 0.9152 and EER = 11.24%; adding SE blocks improved performance to AUC = 0.9581 and EER = 7.52%; and the complete ESNN with Attn-Adam achieved AUC = 0.9892 and EER = 4.20%. The model was converted to TensorFlow Lite using post-training Int8 quantization, with the reported model size decreasing from approximately 45 MB to 11 MB. The mobile authentication workflow uses serial face-then-fingerprint verification and Android Keystore/Trusted Execution Environment support for cryptographic binding. The findings indicate that combining residual feature learning, channel attention, and temporal gradient attention can improve facial verification performance while supporting deployment in mobile financial environments.