Hybrid vision transformer framework for congenital heart disease diagnosis

Congenital heart disease is one of the most common birth defects and has been a source of significant morbidity and mortality among infants. Early diagnosis allows for early intervention and better patient outcomes. Traditional methods are effective yet have limitations such as high costs, dependency on expert knowledge, and noise. Recent developments in deep learning showcase the potential to address these limitations and automate diagnosis with high accuracies. This work proposes the use of a hybrid vision transformer (H_ViTs) model for the diagnosis of congenital heart disease based on heart sounds. The proposed vision transformer-based architecture leverages both local feature extraction with convolutional neural networks and global pattern modeling by vision transformer. Thus, the model is able to capture local dependencies with convolutional neural networks and global dependencies with vision transformers in heart sounds’ spectrogram representations. This study utilized a recently released open-source ZCHSound dataset of children’s pediatric heart sound recordings. To evaluate this model, we use accuracy, precision, recall, F-1 measure, and the confusion matrix as evaluation measures. Our proposed model obtained better results as compared to previously reported results on the same dataset. However, we notice comparatively poor results for multi-class classification, particularly for classes with limited data. In summary, the proposed hybrid model effectively detects congenital heart disease, though additional efforts are needed to improve robustness against class imbalance and noise.