Image Segmentation Method Based on Cross-Modal Attention Fusion

In image segmentation tasks, Segment Anything Model possesses excellent zero-sample generalisation capability. However, despite its strong global semantic modelling capability, SAM shows significant unevenness in the confidence distribution of the generated masks and insufficient ability to capture local structural information in the face of target edges, object occlusion, or severe background interference. For this reason, this paper proposes a lightweight structure enhancement strategy for lowconfidence regions, aiming to improve SAM's ability to perceive detailed structures in complex scenes. Convolutional neural network (CNN) is introduced as an auxiliary structural branch to extract local detail features with high spatial resolution for the above regions, complementing the ViT encoder's deficiency in structural modelling; and a cross-modal attention fusion module is designed to ensure the effectiveness of structural compensation and semantic consistency. The confidence-guided structure enhancement strategy proposed in this paper significantly improves the detail perception and semantic consistency of the Segment Anything model in complex scenes while maintaining high efficiency, and lays a theoretical and practical foundation for future research on the fusion of multimodal perception and lightweight visual enhancement models.