Lesion-Grounded Content-Based Retrieval of CT Findings Using Foundation Model Patch Embeddings for Case Review

Abstract Background Computed tomography (CT) education benefits from prior examinations containing related localized findings. Whole-volume retrieval can be driven by global anatomy or unrelated findings, while database-specific retrieval models limit reuse. This study evaluates whether patch tokens from a frozen volumetric CT foundation encoder can support a reusable lesion-grounded retrieval framework without task-specific retrieval fine-tuning. Methods Spatial patch tokens were pooled into lesion representations linked to their parent examination, annotation, and source finding text. ReXGroundingCT served as the demonstration benchmark. The primary evaluation used FlexiCT-3D to search 14,359 reference lesions with 407 public-validation queries from 50 CT examinations. Exact masks, bounding boxes, and loose boxes tested sensitivity to query-region precision. Retrieval was evaluated against ontology-based relevance, analytic chance expectations, and annotation-informed descriptor controls. Results For exact-mask queries, exact-subcategory Recall@1 was 0.484, more than three times the analytic chance expectation of 0.154, while Recall@10 was 0.848 versus 0.729. Bounding-box and loose-box queries gave Recall@10 of 0.840 and 0.821, respectively, but less precise query regions lowered early-rank performance. Under both box policies, embedding retrieval outperformed the descriptor-only control. Conclusions A reusable lesion-retrieval layer can be built from frozen volumetric foundation features without task-specific retrieval training. The layer retains links to source finding text without using text for ranking and can organize compatible medical-image databases into lesion-centered ranked lists and localized review panels for educational case comparison.

Authors

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-10854370/v1
Primary Topic
Image Retrieval and Classification Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Lesion-Grounded Content-Based Retrieval of CT Findings Using Foundation Model Patch Embeddings for Case Review

Zhongsheng Eric Yu
Research Square
Image Retrieval and Classification Techniques
preprint

Lesion-Grounded Content-Based Retrieval of CT Findings Using Foundation Model Patch Embeddings for Case Review

Zhongsheng Eric Yu
preprint en

Abstract

Abstract Background Computed tomography (CT) education benefits from prior examinations containing related localized findings. Whole-volume retrieval can be driven by global anatomy or unrelated findings, while database-specific retrieval models limit reuse. This study evaluates whether patch tokens from a frozen volumetric CT foundation encoder can support a reusable lesion-grounded retrieval framework without task-specific retrieval fine-tuning. Methods Spatial patch tokens were pooled into lesion representations linked to their parent examination, annotation, and source finding text. ReXGroundingCT served as the demonstration benchmark. The primary evaluation used FlexiCT-3D to search 14,359 reference lesions with 407 public-validation queries from 50 CT examinations. Exact masks, bounding boxes, and loose boxes tested sensitivity to query-region precision. Retrieval was evaluated against ontology-based relevance, analytic chance expectations, and annotation-informed descriptor controls. Results For exact-mask queries, exact-subcategory Recall@1 was 0.484, more than three times the analytic chance expectation of 0.154, while Recall@10 was 0.848 versus 0.729. Bounding-box and loose-box queries gave Recall@10 of 0.840 and 0.821, respectively, but less precise query regions lowered early-rank performance. Under both box policies, embedding retrieval outperformed the descriptor-only control. Conclusions A reusable lesion-retrieval layer can be built from frozen volumetric foundation features without task-specific retrieval training. The layer retains links to source finding text without using text for ranking and can organize compatible medical-image databases into lesion-centered ranked lists and localized review panels for educational case comparison.

Research Square
Image Retrieval and Classification Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon