Clustering Multimodal Ensemble Learning for Predicting Gastric Cancer Neoadjuvant Chemotherapy Efficacy

Accurately predicting the response to neoadjuvant chemotherapy (NCT) is crucial for gastric cancer treatment planning. Multimodal diagnostic methods enhance prediction accuracy by integrating images and clinical features, but they often overlook intra-class differences, causing feature entanglement, and single models struggle to capture dataset diversity. To address these issues, we propose Clustering Multimodal Ensemble Learning (CMEL) framework, which aligns and fuses image features from different domains with clinical features and performs decision fusion using an ensemble model. Specifically, we design the clustering process using Siamese network and memory bank to partition images in feature space and introduce contrastive clustering loss to enhance clustering properties and reduce feature entanglement across domains. We design Hierarchical Feature Alignment Fusion (HFAF) module to generate multi-level image features at different depths and then separately align and fuse them with clinical features, providing diverse features to cover variations in the dataset. We design Dynamic Multi-Classifier Decision Fusion (DMDF) module to predict on fused features using a gating mechanism supervised by pseudo-labels for dynamic weights, enhancing prediction across different domains. On our collected the Gastric Cancer Chemotherapy Response (GCCR) dataset, CMEL achieves an AUC of 76.09%, accuracy of 72.00%, PPV of 86.02%. These results demonstrate the feasibility of joint image and clinical data in predicting NCT response and provide valuable insights for gastric cancer treatment. The code is available at https://github.com/WHX0259/CMEL.