Alternate Interaction Multi-Task Hierarchical Model for Medical Images

Deep learning models in medical image segmentation and classification tasks can automatically extract features and perform high-performance inference, thereby assisting doctors in achieving efficient and accurate automated decision support. However, these models are typically trained for a single task, which leads to limitations such as ignoring task relevance and poor scalability. In response to the above problems, we hope to apply one model to different tasks to enhance the model’s scalability. Hence, this paper proposes an Alternating Multi-Task Hierarchical Network (AMTH-Net) for medical image segmentation and classification. The model is divided into three hierarchical modules: Pathological Region Clarity (PRC) serves as an auxiliary module to improve segmentation and classification capabilities, the Multi-Resolution Attention (MRA) segmentation module focuses on image information at different resolution levels through deep supervision to enhance segmentation accuracy, and the Cascading Multi-Scale Information (CMSI) classification module employs a cascading multi-scale mechanism to gradually integrate discrete information from different network layers, thereby enhancing classification performance. Additionally, we propose a novel Alternating Interaction Loss (AI-Loss) based on a Multi-Level Gradient Information Feedback (MGIF) algorithm to further improve the model’s segmentation and diagnostic performance. We validated our approach on two datasets: the public COVID CXR dataset and our newly proposed F_BUSI breast ultrasound dataset. Experimental results demonstrate that AMTH-Net achieves excellent performance in both segmentation and classification, and it outperforms existing methods in terms of segmentation and classification capabilities.