Resource optimized neural architecture search for automated 3D prostate cancer segmentation from diffusion weighted MRI using weight sharing U-Net framework

Abstract Introduction Prostate cancer is the second most frequently diagnosed malignancy in men worldwide, with more than 1.46 million new cases reported in past years. Accurate prostate cancer localization from Diffusion-Weighted (DWI) MRI remains challenging due to issues such as heterogeneous tumor morphology, anisotropic volumetric resolution, limited annotated datasets, and high inter-patient variability. However, accurate segmentation of prostate cancer from DWI-MRI is very much essential for early diagnosis, treatment planning, and disease monitoring. Background Although 3D U-Net architectures are widely used in the literature for volumetric medical image segmentation; yet, they depend on manually selected architectural configurations, which makes them suboptimal across heterogeneous clinical datasets. Further, conventional Neural Architecture Search (NAS) methods are computationally expensive and unfeasible for resource-constrained medical imaging environments. Methodology This article proposes a Resource-Optimized NAS-Driven U-Net framework for automated 3D prostate tumor segmentation, which integrates a weight-sharing supernet with two-layer LSTM controller trained using Reinforce policy-gradient algorithm for efficiently exploring the structured optimized search space of $$\approx $$ 5.1M candidate architectures. The primary contribution of this work is the development of a resource-efficient NAS-based segmentation framework compared to the baseline, which simultaneously addresses the methodological challenge of computationally expensive architecture search and the clinical challenge of accurate prostate tumor segmentation from DWI-MRI. Results The framework is evaluated on 34 volumetric DWI scans ( $$224\times 224\times 35$$ voxels) collected from Clinical patient dataset of Kasturba Hospital, Manipal. NAS is conducted over 300 episodes, by sampling 20 child architectures per episode, which demonstrates stable convergence of controller, with entropy gradually decreasing and reward progression improving from 0.02 to 0.54. This optimized architecture achieves the maximum validation Dice score of 0.6667 within 50 episodes ( $$\approx $$ 8 h) and outperforms other architectures and search configurations, which yielded lower Dice scores despite higher computational cost. The results demonstrate that the proposed resource-optimized NAS framework efficiently discovers the compact, high-performing 3D segmentation architectures and shows its potential for future clinical prostate cancer delineation.