DeepCombine: Multimodal Person Recognition

The paper explores multimodal biometric recognition by integrating face, iris, and fingerprint data to enhance person identification accuracy and robustness. We compare the effectiveness of late fusion with traditional unimodal approaches, demonstrating that modality fusion significantly improves classification performance. A combined dataset was created by merging CelebFaces, Casia-Iris-Thousand, and Casia Fingerprint datasets, with preprocessing steps to address demographic and sample imbalances. We evaluate feature extraction models, including EfficientNet and FaceNet, and employ machine learning classifiers such as k-Nearest Neighbors (kNN) and Random Forest (RF). Our experiments show that kNN, combined with Principal Component Analysis (PCA) for dimensionality reduction, achieves perfect classification accuracy when all three modalities are fused. While advanced fusion techniques like EmbraceNet were tested, they failed to outperform simpler late fusion methods. These findings highlight the effectiveness of straightforward and interpretable multimodal fusion techniques for biometric recognition, providing a balance between performance and model complexity.