Heartid: A Deep Learning Model for Biometrics Using Multimodal ECG and PPG Signals

Biometrics is one of the most prominent sources of knowledge for human identification in the rapidly advancing field of cybersecurity. Among various types of biometric information, electrocardiogram (ECG) and photoplethysmography (PPG) signals gained considerable attention due to its inherent resistance to spoofing. However, existing researches are predominantly conducted in controlled environments, where ECG or PPG signals are susceptible to interference from motion and noise in real-world scenarios, resulting in suboptimal model robustness. Additionally, the few multimodal ECG identification studies mainly based on different conversion methods of ECG signal, resulting in information redundancy between modalities. To address these, we propose a lightweight identity recognition model based on cross-domain multimodal feature fusion of ECG and PPG signals. It incorporates an enhanced MultiRes block that forwards upsampled data for multi-resolution analysis of signal features, and employs spatial pyramid pooling for multi-scale feature extraction. Using this network structure, we separately extract and fuse features from ECG and PPG signals, then perform identity recognition based on the fused features. Extensive experiments were conducted to evaluate the proposed algorithm. The proposed fusion model reaches a high accuracy of 97.26%, improving by 17.06% and 40.48% compared to single-modal ECG and PPG identification models, respectively. By visualizing the fused features alongside single-modal ECG and PPG features, we intuitively explain the effectiveness of multimodal feature fusion for feature extraction, offering a new approach for multimodal biometric recognition.