A transformer-enhanced U-Net with FiLM-driven clinical conditioning for multi-task breast ultrasound lesion analysis

Abstract Breast cancer remains one of the leading causes of cancer-related mortality among women worldwide. Early and accurate detection of breast lesions using non-invasive imaging techniques is crucial for improving timely diagnosis, treatment outcomes, and patient prognosis. This study aims to develop and evaluate hybrid deep learning models for breast-lesion segmentation and classification in ultrasound images using patient-level cross-validation. Challenges such as low contrast, imaging artifacts, and irregular lesion morphology can limit accurate diagnosis, highlighting the need for advanced deep learning approaches for reliable breast-lesion analysis. Recent developments in deep learning, particularly hybrid architectures that combine convolutional and transformer representations, have shown promise in addressing these constraints. We present three Hybrid Deep Learning (HDL) models. Three complementary deep learning models were evaluated for breast lesion classification and segmentation using the Breast-Lesions-USG dataset. Model 1 employed DoubleHeadUNet with a ResNet-34 backbone, Model 2 combined a Swin V2 Transformer with a U-Net decoder, and Model 3 integrated a Hybrid Transformer-FiLM U-Net to incorporate clinical metadata. Patient-level five-fold cross-validation with strict patient separation and nested hyperparameter selection was employed. Model 1 achieved the highest accuracy (82.94%) and AUC (87.70%), while Model 3 demonstrated the highest malignant recall (81.63%) and strongest segmentation performance (Dice: 0.901; IoU: 0.825). Model 2 achieved an accuracy of 79.76% and an AUC of 87.54%. Statistical analysis indicated that most accuracy differences were not significant, supporting the complementary strengths of the three models rather than identifying a universally superior approach. Comparisons with previous studies were considered indicative due to differences in datasets and evaluation protocols.