DTA-Net: Dual-Task Attention Network for Medical Image Segmentation and Classification

Medical image segmentation and classification are fundamental tasks in computer-aided diagnosis, where accurate segmentation plays a key role in identifying disease-related features and regions of interest, thus aiding subsequent classification. In this paper, we propose a novel Dual-Task Attention Network (DTA-Net), which simultaneously generates high-quality segmentation masks and performs image classification by incorporating Electronic Medical Records (EMR). The DTA-Net architecture introduces an innovative Shared Proxy Attention Module (SPAM), which leverages a shared mapping function to effectively encode the attention weights of both queries and keys for spatial and channel attention. This dual attention mechanism facilitates complementary learning and interdependence between spatial and channel features. Notably, the proxy projection mechanism within SPAM significantly reduces the computational complexity of spatial attention to a linear level. We validated our approach on three datasets: MSD Prostate for prostate segmentation, the RICORD dataset for lung segmentation, and the iCTCF COVID-19 dataset for both segmentation and classification tasks. Experimental results demonstrate that the proposed network achieves promising performance across all datasets. The source code is available at: https://github.com/HaifengQi/DTA-Net.