Know your Heart : A multimodal pipeline for effective detection and inference of heart diseases

The accurate and timely diagnosis of heart diseases is crucial for effective patient management and treatment. Traditional diagnostic methods often rely on singular modalities, which may lack the comprehensive insight needed for accurate diagnosis. The approach planned in this paper leverages the strengths of three modalities to provide more robust and holistic diagnostic tool. The proposed pipeline begins with acquisition of synchronized Color Doppler video, Electrocardiogram(ECG), and Electroencephalogram(EEG) signals. The Color Doppler Echo-cardiography offers detailed visualization of blood flow dynamics and heart valve function, which is crucial for assessing valvular heart diseases (VHD). ECG provides essential information on the electrical activity of the heart, revealing different arrhythmia and other cardiac conditions. EEG, atypically associated with brain activity, is incorporated to capture autonomic nervous system responses which may reflect underlying cardiac issues. Advanced pre-processing techniques are applied to each modality to ensure data quality and noise reduction. Feature extraction from these modalities is followed by a feature fusion stage, where machine-learning (ML) algorithms are employed to integrate the diverse data streams. This fusion is designed to exploit the complementary nature of the modalities with enhancing the overall diagnostic accuracy. The multimodal pipeline’s efficacy is validated using a dataset comprising patients with diagnosed heart diseases including VHD also. The results demonstrate significant improvements in diagnostic accuracy compared to unimodal approaches. The integration of Color Doppler, ECG, and EEG provides a comprehensive view of the cardiac function, capturing both structural and functional abnormalities. By combining the visual insights of echocardiography with the electrical activity captured by ECG and EEG, the proposed and designed method offers a powerful tool for early and accurate detection of heart diseases.