Multimodal-multiscale hybrid fusion network for automated esophageal cancer diagnosis

Accurate and early diagnosis of esophageal cancer (EC) remains challenging due to heterogeneous imaging characteristics and the complexity of integrating multimodal data. This study proposes a multimodal-multiscale hybrid fusion network (MHF-Net), which combines domain-driven handcrafted features (HFs) and deep multiscale features (DFs) from multimodal data for the automated diagnosis of EC. MHF-Net uses a dilated-inception block to extract multiscale representations at varied dilation rates and concatenates these DF with domain-specific features via dense connections. A convolutional block attention module further refined spatial and channel-wise features, leveraging global average pooling. The grasshopper optimization algorithm optimized fusion weights and hyperparameters, enhancing hybrid feature integration and overall model robustness. Evaluated across five diverse datasets, MHF-Net achieves state-of-the-art performance (accuracy = 0.970 ± 0.014; F1 score = 0.960 ± 0.017). This study demonstrates clinical applicability with strong potential for future enhancement through the integration of multimodal biomarkers for enhanced diagnostic accuracy.