Automated 3D segmentation of human vagus nerve fascicles and epineurium from micro-computed tomography images using anatomy-aware neural networks

<i>Objective.</i>Precise segmentation and quantification of nerve morphology from imaging data are critical for designing effective and selective peripheral nerve stimulation (PNS) therapies. However, prior studies on nerve morphology segmentation suffer from important limitations in both accuracy and efficiency. This study introduces a deep learning approach for robust and automated three-dimensional (3D) segmentation of human vagus nerve fascicles and epineurium from high-resolution micro-computed tomography (microCT) images.<i>Methods.</i>We developed a multi-class 3D U-Net to segment fascicles and epineurium that incorporates a novel anatomy-aware loss function to ensure that predictions respect nerve topology. We trained and tested the network using subject-level five-fold cross-validation with 100 microCT volumes (11.4<i>μ</i>m isotropic resolution) from cervical and thoracic vagus nerves stained with phosphotungstic acid from five subjects. We benchmarked the 3D U-Net's performance against a two-dimensional (2D) U-Net using both standard and anatomy-specific segmentation metrics.<i>Results.</i>Our 3D U-Net generated high-quality segmentations (average Dice similarity coefficient: 0.93). Compared to a 2D U-Net, our 3D U-Net yielded significantly better volumetric overlap, boundary delineation, and fascicle instance detection. The 3D approach reduced anatomical errors (topological and morphological implausibility) by 2.5-fold, provided more consistent inter-slice boundaries, and improved detection of fascicle splits/merges by nearly 6-fold.<i>Significance.</i>Our automated 3D segmentation pipeline provides anatomically accurate 3D maps of peripheral neural morphology from microCT data. The automation allows for high throughput, and the substantial improvement in segmentation quality and anatomical fidelity enhances the reliability of morphological analysis, vagal pathway mapping, and the implementation of realistic computational models. These advancements provide a foundation for understanding the functional organization of the vagus and other peripheral nerves and optimizing PNS therapies.