A robust deep learning approach for segmenting cortical and trabecular bone from 3D high resolution µCT scans of mouse bone

Recent advancements in deep learning have significantly enhanced the segmentation of high-resolution microcomputed tomography (µCT) bone scans. In this paper, we present the dual-branch attention-based hybrid network (DBAHNet), a deep learning architecture designed for automatically segmenting the cortical and trabecular compartments in 3D µCT scans of mouse tibiae. DBAHNet's hierarchical structure combines transformers and convolutional neural networks to capture long-range dependencies and local features for improved contextual representation. We trained DBAHNet on a limited dataset of 3D µCT scans of mouse tibiae and evaluated its performance on a diverse dataset collected from seven different research studies. This evaluation covered variations in resolutions, ages, mouse strains, drug treatments, surgical procedures, and mechanical loading. DBAHNet demonstrated excellent performance, achieving high accuracy, particularly in challenging scenarios with significantly altered bone morphology. The model's robustness and generalization capabilities were rigorously tested under diverse and unseen conditions, confirming its effectiveness in the automated segmentation of high-resolution µCT mouse tibia scans. Our findings highlight DBAHNet's potential to provide reliable and accurate 3D µCT mouse tibia segmentation, thereby enhancing and accelerating preclinical bone studies in drug development. The model and code are available at https://github.com/bigfahma/DBAHNet .