Seg & Ref: a newly developed toolset for artificial intelligence-powered segmentation and interactive refinement for labor-saving three-dimensional reconstruction

Traditional three-dimensional (3D) reconstruction is labor-intensive owing to manual segmentation; this can be addressed by developing artificial intelligence (AI)-driven automated segmentation. However, it is limited by a lack of user-friendly tools for morphologists. We present a workflow for 3D reconstruction using our AI-powered segmentation tool. Specifically, we developed an interactive toolset, 'Seg & Ref', to overcome the abovementioned challenges by enabling AI-powered segmentation and easy mask editing without requiring a command-line setup. We demonstrated a 3D reconstruction workflow using serial sections of a Carnegie Stage 15 human embryo. Automated segmentation (Step 1) was performed using the graphical user interface, 'SAM2 GUI for Img Seq', which utilizes the Segment Anything Model 2 and supports interactive segmentation through a web-based interface. Users specify target structures via box prompts, and the results are propagated across all images for batch segmentation. The segmentation masks were reviewed and corrected (Step 2) using 'Segment Editor PP', a PowerPoint-based tool enabling interactive mask refinement. Finally, the corrected masks were imported into the 3D Slicer application for reconstruction (Step 3). Our 3D reconstruction visualized key structures, including the spinal cord, veins, aorta, mesonephros, gut, heart, trachea, liver and peritoneal cavity. The reconstructed models accurately represented their spatial relationships and morphologies. This provides a labor-saving approach for 3D reconstruction workflows owing to their optimization for serial sections, versatility and accessibility without programming expertise. Therefore, morphological research can be enhanced by precise segmentation using intuitive and user-friendly interfaces of 'Seg & Ref'.