Prostate-Specific Membrane Antigen PET/CT Segmentation Guided by Vision Language Models Embedding Clinical Report Semantics

Whole-body prostate-specific membrane antigen (PSMA) PET/CT segmentation remains challenging: manual contouring is unsustainable, and image-only artificial intelligence models often misclassify physiologic uptake as malignant, causing high false-positive rates. These limitations underscore the need for segmentation strategies that incorporate the nuclear medicine expertise encoded in PET/CT reports but not used by current models. We investigated whether routine PET/CT reports improve automated lesion segmentation for prostate cancer. Methods: We developed a language-guided 3-dimensional SegResNet fusing report text with 68Ga-PSMA-11 PET/CT image features, encoding report semantics via 4 strategies (BiomedVLP, RadGraph, GPT-raw, GPT-engineered) compared with an image-only model (baseline). Of 320 patients who underwent 68Ga-PSMA-11 PET/CT, 297 studies were available; 13 were excluded for missing reports or incomplete image-mask pairs, leaving 284 for analysis, split at the patient level into 243 development cases and a fixed 41-case held-out test set. Metrics included Dice, intersection over union (IoU), precision, recall, average symmetric surface distance, 95th percentile Hausdorff distance, surface Dice similarity at 5 mm, and lesion-level false positives, using 3 matching criteria (any overlap; 1-to-1 maximum IoU matching, IoU ≥ 0.5; and correct localization of the SUVmax voxel). Significance was assessed using paired 1-sided Wilcoxon tests with Holm adjustment. Results: Report guidance consistently reduced false positives and improved whole-mask boundary agreement. False-positive lesions were reduced by nearly 4 per patient with BiomedVLP and by approximately 3 per patient with RadGraph and GPT-raw. Average symmetric surface distance decreased from 105.4 ± 62.5 mm (image only) to 23.4 ± 18.2 mm with BiomedVLP, 36.9 ± 22.4 mm with RadGraph, and 36.3 ± 26.4 mm with GPT-raw; 95th percentile Hausdorff distance improved from 455.0 ± 169.3 mm to 123.8 ± 80.3 mm with BiomedVLP, and surface Dice similarity at 5 mm increased from 0.638 ± 0.172 to 0.705 ± 0.181. However, when these surface metrics were recomputed on matched true-positive lesions only using the same 3 criteria, no language-guided model differed significantly from the image-only model for any surface metric (all Holm-adjusted P = 1.0), indicating that the whole-mask improvements were attributable to false-positive suppression rather than genuine gains in contour accuracy on correctly detected lesions. Precision increased for BiomedVLP (0.099) and RadGraph (0.060), with a modest decrease in voxel-level recall (−0.051, BiomedVLP), reflecting a precision-recall trade-off from suppression of spurious uptake. Dice improved by 0.032 but was borderline significant; IoU increased for BiomedVLP (0.036). Lesion-level sensitivity changed by no more than 0.04 for all models and did not reach significance. Total lesion PSMA remained highly correlated with manual measurements (r ≥ 0.85), whereas PSMA tumor volume correlations were more variable across models (r = 0.37–0.70). Conclusion: The integration of PET/CT report semantics into automated PSMA PET/CT segmentation substantially reduced false-positive lesions and preserved total lesion PSMA agreement with manual measurements; a matched true-positive-only reanalysis indicated that the accompanying whole-mask boundary-agreement gains reflected false-positive suppression rather than improved contour accuracy on correctly detected lesions. This approach may reduce manual cleanup and support more reliable quantitative workflows, although precision gains should be weighed against a modest recall reduction. External validation, extension of the true-positive-only analysis to independent cohorts, and PSMA-specific language pretraining are warranted.

Authors

Institutions

Publication Details

Journal
Journal of Nuclear Medicine
Published
2026-11-12
DOI
https://doi.org/10.2967/jnumed.126.272702
Primary Topic
Medical Image Segmentation Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Prostate-Specific Membrane Antigen PET/CT Segmentation Guided by Vision Language Models Embedding Clinical Report Semantics

Elinaz Hosseinzadeh, Arman Rahmim, Ghasemali Divband, Rezvan Samimi et al.
Journal of Nuclear Medicine
Medical Image Segmentation Techniques
article

Prostate-Specific Membrane Antigen PET/CT Segmentation Guided by Vision Language Models Embedding Clinical Report Semantics

Elinaz Hosseinzadeh, Arman Rahmim, Ghasemali Divband, Rezvan Samimi, Nourhan Bayasi, Zohreh Adinehpour, Fereshteh Yousefirizi, Mahan Pouromidi, Shaghayegh Shafiee Dastgerdi, Babak Nickkholgh
article en

Abstract

Whole-body prostate-specific membrane antigen (PSMA) PET/CT segmentation remains challenging: manual contouring is unsustainable, and image-only artificial intelligence models often misclassify physiologic uptake as malignant, causing high false-positive rates. These limitations underscore the need for segmentation strategies that incorporate the nuclear medicine expertise encoded in PET/CT reports but not used by current models. We investigated whether routine PET/CT reports improve automated lesion segmentation for prostate cancer. Methods: We developed a language-guided 3-dimensional SegResNet fusing report text with 68Ga-PSMA-11 PET/CT image features, encoding report semantics via 4 strategies (BiomedVLP, RadGraph, GPT-raw, GPT-engineered) compared with an image-only model (baseline). Of 320 patients who underwent 68Ga-PSMA-11 PET/CT, 297 studies were available; 13 were excluded for missing reports or incomplete image-mask pairs, leaving 284 for analysis, split at the patient level into 243 development cases and a fixed 41-case held-out test set. Metrics included Dice, intersection over union (IoU), precision, recall, average symmetric surface distance, 95th percentile Hausdorff distance, surface Dice similarity at 5 mm, and lesion-level false positives, using 3 matching criteria (any overlap; 1-to-1 maximum IoU matching, IoU ≥ 0.5; and correct localization of the SUVmax voxel). Significance was assessed using paired 1-sided Wilcoxon tests with Holm adjustment. Results: Report guidance consistently reduced false positives and improved whole-mask boundary agreement. False-positive lesions were reduced by nearly 4 per patient with BiomedVLP and by approximately 3 per patient with RadGraph and GPT-raw. Average symmetric surface distance decreased from 105.4 ± 62.5 mm (image only) to 23.4 ± 18.2 mm with BiomedVLP, 36.9 ± 22.4 mm with RadGraph, and 36.3 ± 26.4 mm with GPT-raw; 95th percentile Hausdorff distance improved from 455.0 ± 169.3 mm to 123.8 ± 80.3 mm with BiomedVLP, and surface Dice similarity at 5 mm increased from 0.638 ± 0.172 to 0.705 ± 0.181. However, when these surface metrics were recomputed on matched true-positive lesions only using the same 3 criteria, no language-guided model differed significantly from the image-only model for any surface metric (all Holm-adjusted P = 1.0), indicating that the whole-mask improvements were attributable to false-positive suppression rather than genuine gains in contour accuracy on correctly detected lesions. Precision increased for BiomedVLP (0.099) and RadGraph (0.060), with a modest decrease in voxel-level recall (−0.051, BiomedVLP), reflecting a precision-recall trade-off from suppression of spurious uptake. Dice improved by 0.032 but was borderline significant; IoU increased for BiomedVLP (0.036). Lesion-level sensitivity changed by no more than 0.04 for all models and did not reach significance. Total lesion PSMA remained highly correlated with manual measurements (r ≥ 0.85), whereas PSMA tumor volume correlations were more variable across models (r = 0.37–0.70). Conclusion: The integration of PET/CT report semantics into automated PSMA PET/CT segmentation substantially reduced false-positive lesions and preserved total lesion PSMA agreement with manual measurements; a matched true-positive-only reanalysis indicated that the accompanying whole-mask boundary-agreement gains reflected false-positive suppression rather than improved contour accuracy on correctly detected lesions. This approach may reduce manual cleanup and support more reliable quantitative workflows, although precision gains should be weighed against a modest recall reduction. External validation, extension of the true-positive-only analysis to independent cohorts, and PSMA-specific language pretraining are warranted.

Journal of Nuclear Medicine
BC Cancer Agency (CA), University of British Columbia (CA), University of Isfahan (IR), Shahid Beheshti University (IR), Shahid Beheshti University of Medical Sciences (IR), McMaster University (CA)
Good health and well-being
Medical Image Segmentation Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon