Instance-based Aortic Valve Segmentation and Landmark Detection on RGB Images

Abstract For surgical aortic valve repair or replacement, intraoperative assessment of valve function and geometry is essential. The existing approaches are limited by nonphysiological conditions and lack objective, automated measurement. Image-based methods have been proposed to address this, but existing approaches primarily rely on semantic segmentation, which is restricted to regular tricuspid valves and does not generalize to pathological configurations. In this study, we extend semantic segmentation to an instanceaware and landmark-based formulation. This enables the identification of individual valve cusps. We employ a Mask RCNN framework and augment it with an additional heatmapbased landmark prediction head. The approach is evaluated on a dataset of porcine aortic valves. The efficacy of various ResNet and Swin Transformer backbones is evaluated, and a refined model is subsequently developed to enhance performance. The proposed method demonstrates competitive segmentation performance (mIoU 0.8794), which is only marginally below the dataset benchmark (0.9158), while enabling landmark detection with recall values of up to 0.74. The findings of this study demonstrate the feasibility of combining instance segmentation and landmark detection for more comprehensive valve analysis.