TF-crossnet: a cross-modal attention fusion network for cardiovascular disease classification using pcg and ecg signals

Electrocardiogram (ECG) and phonocardiogram (PCG) have emerged as crucial non-invasive and portable diagnostic modalities for early cardiovascular disease (CVD) screening. Despite the individual merits of these signal modalities in CVD detection, significant challenges persist, including insufficient inter-modal interaction and suboptimal weight allocation. To address these critical limitations, we proposed a novel Time-Frequency Cross-Modal Attention Fusion Network (TF-CrossNet) designed for precise early CVD diagnosis. The proposed network employs a dual-path multiscale residual structure to extract key time-frequency domain features from PCG and ECG signals, comprehensively capturing multiscale information. Leveraging the intrinsic electro-mechanical coupling relationship of the heart, a bidirectional mutual enhancement attention module is introduced to capture interactive morphological information between PCG and ECG signals, enabling feature-level signal complementation and enhancement. Furthermore, an adaptive fusion strategy based on Bayesian decision theory is developed, establishing a mapping relationship between confidence levels and loss functions to dynamically optimize modal weight allocation. Validated on the 2016 PhysioNet/CinC dataset, the model achieved exceptional performance metrics: 93.13% accuracy, 97.7% specificity, and 98% area under the curve (AUC). Furthermore, comprehensive noise robustness experiments demonstrate that TF-CrossNet maintains superior performance under various noise conditions, achieving an average robustness index of 94.20% compared to existing methods, validating its practical applicability in clinical environments. The superior effectiveness of the proposed approach in CVD classification, providing a novel technological pathway for non-invasive and precision CVD diagnosis.