TF-Net: Triple Fusion Net for Medical Image Segmentation

Lesion segmentation plays a crucial role in various medical image analyses, which not only improves the efficiency in clinical diagnosis but also assists in detecting early symptoms of various diseases. Most existing studies focus on directly extracting lesion information from specific types of medical images with pre-trained weights, often neglecting the underlying topological and pathological causes which lead to these lesions. Furthermore, they overlook to capture general anatomical features among lesions, which are related to the distribution of lesions, and thus the model is poorly generalized in different medical datasets. Inspired by these insights, we propose a Triple Fusion Net (TF-Net), a network structure divided into three branches: left, middle and right. The left and right branches are designed to extract lesion features and associated topological style features within various medical images, respectively. And these features are further fused and modeled in the middle branch. The proposed structure of triple branches for features fusing effectively learns multi-feature information and improves the performance of TF-Net. And our work experiments validate various feature fusion methods in the middle branch, including channel-wise concatenation, element-wise addition, attention gate, and transformer encoder block. Without using pre-trained weights in our network, the transformer encoder block performs best on some tasks of DDR and surpasses other pre-trained models. Channel concatenation exhibits performance close to other pre-trained models in both the IDRiD, Kvasir-Seg and TN3K. Attention gate fusion also shows competitive results in thyroid ultrasound segmentation. Our approach, leveraging a unique network structure and four different feature fusion methods, demonstrates remarkable generality across a spectrum of medical image segmentation tasks.