Stage-wise mutual attention network for diagnostic support of chronic ischaemic heart disease using multimodal medical information

Chronic ischaemic heart disease (CIHD) is a leading cause of death worldwide. Contributing factors include lifestyle-related diseases, such as diabetes and hypertension, as well as genetic predispositions. The diagnosis of CIHD requires physicians to perform multiple tests, including highly invasive tests, which impose a significant burden on patients. To address this issue, a diagnostic support network for CIHD has been proposed, which uses multimodal medical information obtained solely from non-invasive tests as input. However, in this system, feature integration from different modalities is performed by concatenating each feature, which may not fully account for the diagnostic relevance of each input. In this study, we propose a new diagnostic support network for CIHD that integrates multimodal features using an attention mechanism. Moreover, we introduce a stage-wise integration approach in which different feature vectors are progressively combined, two at a time. This allows features from different modalities to be gradually integrated while preserving diagnostically relevant information. We confirmed that the proposed method outperforms conventional approaches in classifying healthy subjects and patients using non-invasive multimodal data. Furthermore, we demonstrated that the classification performance improves when integrating the three modalities step-by-step, starting from pairs of closely related modalities rather than merging all three modalities simultaneously. • We propose Stage-Wise Mutual Attention Network (SWAN), a diagnostic support network. • SWAN diagnoses chronic ischaemic heart disease using non-invasive multimodal data. • A stage-wise fusion with mutual attention combines modality-specific features. • Step-by-step integration preserves critical diagnostic information across modalities. • Step-by-step modality integration improves accuracy over all-at-once fusion.