Explainable multimodal deep learning using cross-attention fusion of ultrasound and clinical features for PCOS classification

Polycystic Ovary Syndrome (PCOS) is still the most common endocrine disorder among reproductive-aged women, but its proper diagnosis is always disrupted by heterogeneous phenotypes and the uncoordinated application of diagnostic modalities in solitude. Traditional methods mainly depend on either clinical markers or ultrasound imaging, which often results in unstandardized outcomes and less reliability. To break these barriers, this paper proposes an explainable multimodal deep learning model that combines ultrasound images with structured clinical features through a cross-attention fusion mechanism. Ultrasound data are extracted by a ResNet50 backbone as discriminative spatial features, while 41 clinical factors are compressed into 20 important diagnostic factors and embedded by a Transformer Encoder, which can well capture inter-attribute relationships and boost contextual feature representations. The cross-attention module facilitates bidirectional interactions between modalities so that cues from ultrasound can refine clinical feature interpretation and clinical embeddings modulate image-derived features. The fused representation is then passed through a hierarchical classification head to predict the status of PCOS. Interpretability is guaranteed by Grad-CAM + + overlays localizing discriminative ovarian structures in ultrasound images and attention heatmaps quantifying relative importance of clinical attributes. Extensively conducted experiments on a matched dataset of clinical profiles and ultrasound scans show better performance regarding accuracy, AUC, sensitivity, and specificity than unimodal and traditional fusion baselines. The explainability results closely agree with well-known diagnostic markers, stressing the clinical reliability of the proposed paradigm. This generalizable, transparent, and scalable model has huge potential for PCOS classification and more general multimodal disease diagnosis in medical imaging.