Mul2UNet: Deformable Multi-Scale UNet With Decoupled Channel Fusion for Oral Squamous Cell Carcinoma Segmentation

Oral squamous cell carcinoma (OSCC) histopathology analysis faces challenges in automating tumor segmentation, particularly in leveraging multi-channel data and adapting to complex tumor morphology. Existing deep learning methods struggle with (1) inefficient multi-channel utilization due to redundant spectral signals in early fusion, and (2) rigid feature fusion that does not align with OSCC’s complex tumor-stroma interfaces. To address these, we propose Mul2UNet, an enhanced UNet framework for OSCC segmentation that integrates task-driven spectral disentanglement and structure-aware multi-scale fusion. The framework first optimizes channel selection in hybrid RGB-HSV space, suppressing redundant noise and preserving critical features. A deformable multi-scale architecture captures irregular tumor boundaries via gradient-modulated convolutions. Independent processing of selected channels and late fusion enhances nuclear atypia patterns. Evaluated on two datasets against sixteen benchmark models, Mul2UNet outperforms the best baseline, improving IoU by 10.38%, while maintaining computational efficiency, highlighting its potential for improving diagnostic accuracy in digital pathology.