ATFU-Net: adaptive feature refinement and convolutional-style self-attention for medical image segmentation

Accurate segmentation of medical images is crucial for detecting diseases, planning surgeries, and evaluating treatments. This study introduces AdaTransFormerNet, an advanced deep learning framework addressing challenges like contrast variation, noise interference, and multi-scale anatomical structures in medical imaging. The architecture incorporates two key innovations: a ConvFEFormer encoder combining convolutional operations with self-attention mechanisms, and an Adaptive Feature Refinement Module (AFRM) for multi-scale context aggregation.(1)The ConvFEFormer encoder employs deformable convolutions to establish long-range pixel correlations while preserving local spatial relationships. This approach avoids the spatial distortion caused by traditional image patching in vision transformers. (2)The AFRM module dynamically adjusts feature importance through adaptive attention weighting, enhancing the network's ability to focus on clinically significant regions like tumors. Compared to existing models such as TransFuse and FCT, our framework demonstrates superior performance in small-sample medical scenarios through its dual capability of global context modeling and local detail preservation. Evaluations across diverse medical imaging datasets revealed significant improvements. On cardiac MRI (ACDC), the model achieved 97.5% Dice and 98.16% IoU scores. For dermoscopic lesion analysis (ISIC), it attained 95.85% Dice and 97.29% IoU. The retinal fundus dataset (RFI) showed comparable enhancements with 73.52% Dice and 95.32% IoU. These results highlight the model's clinical potential through its adaptive refinement mechanisms and computationally efficient design.