mvEchoSeg: One-shot In-context Learning for Multi-view Echocardiography Segmentation

Echocardiography is the clinical standard for evaluation of cardiac morphology, and function, and providing hemodynamic parameters in patients with known or suspected heart disease. Due to the diversity and complexity of diagnostic tasks, comprehensive interpretation of echocardiography often requires multi-view imaging for integrated metric analysis. Deep learning methods have become the mainstream approach for echocardiography segmentation. However, achieving segmentation of multi-view echocardiography still requires a substantial amount of annotation. To remedy this, we propose a one-shot in-context learning network mvEchoSeg for multi-view echocardiography segmentation. This network requires only one annotated image for each view. Specifically, we propose a Task Prompt Identifier (TPI) module to identify the task type of the image and allocate the most precise task prompt for it, with minimal adaptation of CLIP and few-shot strategies. Additionally, we leverage a unified In-Context Model (ICM) capable of performing a diverse set of echocardiography segmentation tasks automatically. Using fine-tuning and low-rank adapters improved the performance of the pre-trained model, achieving significant results with minimal training cost. Furthermore, we collect a multi-view echocardiography dataset (MVECD) with 8 views to evaluate our method. The results show an improvement of more than 10% in the DICE score compared to the SOTA foundational medical image segmentation models. To our knowledge, this is the first exploration of a one-shot model for multi-view echocardiography segmentation. Our codes and models are available at https://github.com/stellating/mvEchoSeg.