Large language models as developers of quantification software for bone scintigraphy in transthyretin cardiac amyloidosis: a one-shot comparison on a synthetic DICOM phantom

Abstract Purpose The heart-to-contralateral lung (H/CL) ratio is a key semi-quantitative index of bone scintigraphy in transthyretin cardiac amyloidosis (ATTR-CA), but its thresholds were validated mainly for 99m Tc-pyrophosphate. Testing them with diphosphonates requires a fast, reproducible measurement tool. We assessed whether large language models (LLMs) can develop such a tool in one shot, and verified its measurements against a known ground truth. Methods An identical prompt requesting a measurement-only offline web application (native DICOM parser, constrained circular regions of interest [ROIs], H/CL at 1 h and 3 h, retention index [RI], operator-entered Perugini grade, results export, no diagnostic interpretation) was submitted to Claude Opus 5.5, GPT-5.6 Sol, Gemini 3.1 Pro, DeepSeek-V4.1-Flash and Qwen3.8-Omni-Flash. Tools were tested on a synthetic DICOM phantom of planar thoracic images with four cardiac uptake levels and known ground truth, and scored by two nuclear medicine physicians across seven domains. Results Composite scores (0–100) were 97.5 for Claude Opus 5.5 and GPT-5.6 Sol, 91.8 for Gemini 3.1 Pro, 74.4 for Qwen3.8-Omni-Flash and 22.4 for DeepSeek-V4.1-Flash, whose tool could not read DICOM files. The three best tools reproduced total counts exactly and RI within tolerance in all measurements; their H/CL deviations were explained by operator-chosen ROI size. The Qwen3.8-Omni-Flash tool returned plausible but incorrect ratios. Conclusion In this one-shot comparison, three paid-tier LLMs produced research-grade quantification tools, whereas both free-tier outputs were unreliable. AI-generated nuclear medicine software should be verified against a known ground truth before use, even for research.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-04
DOI
https://doi.org/10.21203/rs.3.rs-11403312/v1
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Large language models as developers of quantification software for bone scintigraphy in transthyretin cardiac amyloidosis: a one-shot comparison on a synthetic DICOM phantom

Salvatore Annunziata, Marco De Summa, Guido Maria Meduri, Daniele Pizzuto et al.
Research Square
Artificial Intelligence in Healthcare and Education
preprint

Large language models as developers of quantification software for bone scintigraphy in transthyretin cardiac amyloidosis: a one-shot comparison on a synthetic DICOM phantom

Salvatore Annunziata, Marco De Summa, Guido Maria Meduri, Daniele Pizzuto, Michela Meduri
preprint en

Abstract

Abstract Purpose The heart-to-contralateral lung (H/CL) ratio is a key semi-quantitative index of bone scintigraphy in transthyretin cardiac amyloidosis (ATTR-CA), but its thresholds were validated mainly for 99m Tc-pyrophosphate. Testing them with diphosphonates requires a fast, reproducible measurement tool. We assessed whether large language models (LLMs) can develop such a tool in one shot, and verified its measurements against a known ground truth. Methods An identical prompt requesting a measurement-only offline web application (native DICOM parser, constrained circular regions of interest [ROIs], H/CL at 1 h and 3 h, retention index [RI], operator-entered Perugini grade, results export, no diagnostic interpretation) was submitted to Claude Opus 5.5, GPT-5.6 Sol, Gemini 3.1 Pro, DeepSeek-V4.1-Flash and Qwen3.8-Omni-Flash. Tools were tested on a synthetic DICOM phantom of planar thoracic images with four cardiac uptake levels and known ground truth, and scored by two nuclear medicine physicians across seven domains. Results Composite scores (0–100) were 97.5 for Claude Opus 5.5 and GPT-5.6 Sol, 91.8 for Gemini 3.1 Pro, 74.4 for Qwen3.8-Omni-Flash and 22.4 for DeepSeek-V4.1-Flash, whose tool could not read DICOM files. The three best tools reproduced total counts exactly and RI within tolerance in all measurements; their H/CL deviations were explained by operator-chosen ROI size. The Qwen3.8-Omni-Flash tool returned plausible but incorrect ratios. Conclusion In this one-shot comparison, three paid-tier LLMs produced research-grade quantification tools, whereas both free-tier outputs were unreliable. AI-generated nuclear medicine software should be verified against a known ground truth before use, even for research.

Research Square
Università Cattolica del Sacro Cuore (IT), Università Campus Bio-Medico (IT), Agostino Gemelli University Polyclinic (IT), Campus Bio Medico University Hospital (IT)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon