Towards robust and reliable multi-modal 3D segmentation of multiple sclerosis lesions

• Full 3D multi-modal segmentation model improves MS lesion accuracy. • One to three fine-tuning volumes are sufficient to adapt to an unseen dataset. • One-to-three fine-tuning samples per dataset enables cross-source adaptation under multiple distribution shifts. • Latent Distance Analysis offers a label-free, online, differentiable reliability estimation of model predictions. • Latent Distance Analysis out-of-distribution scores correlate with error metric. Accurate 3D segmentation of multiple sclerosis lesions is critical for clinical practice, yet existing approaches face key limitations: many models rely on 2D architectures or partial modality combinations, while others struggle to generalise across scanners and protocols. Although large-scale, multi-site training can improve robustness, its data demands are often prohibitive. To address these challenges, we propose a 3D multi-modal network that simultaneously processes T1-weighted, T2-weighted, and FLAIR scans, leveraging full cross-modal interactions and volumetric context to achieve state-of-the-art performance across four diverse public datasets. To tackle data scarcity, we quantify the minimal fine-tuning effort needed to adapt to individual unseen datasets and reformulate the few-shot learning paradigm at an “instance-per-dataset” level (rather than traditional “instance-per-class”), enabling the quantification of the minimal fine-tuning effort to adapt to multiple unseen sources simultaneously. Finally, we introduce Latent Distance Analysis , a novel label-free reliability estimation technique that anticipates potential distribution shifts and supports any form of test-time adaptation, thereby strengthening efficient robustness and physicians’ trust.