Intra- and inter-observer agreement of IMPeTUs in multiple myeloma: a blinded repeated-reader study with independent multi-institutional readers

Standardized interpretation of [18F]FDG PET/CT in multiple myeloma remains challenging because reproducibility varies across visual parameters. The Italian Myeloma criteria for PET Use (IMPeTUs) provide a structured framework, but evidence on intra-observer and inter-observer agreement remains limited, particularly across institutions. We prospectively registered patients with histopathologically confirmed multiple myeloma undergoing [18F]FDG PET/CT between March and September 2025. Fifty patients were included according to a prespecified target sample size. After anonymization, three independent nuclear medicine physicians from different institutions evaluated all examinations twice over 10 weeks using a blinded repeated-reader design with software-randomized weekly case allocation and presentation order. Intra-observer agreement was assessed using Cohen’s kappa, inter-observer agreement using Krippendorff’s alpha and pairwise kappa, and differences between reading rounds using permutation-based methods. An exploratory whole-patient binary metabolic status analysis using a Deauville score 4 threshold was also performed. Inter-observer agreement was highest for Lytic Lesion Group (α = 0.683 and 0.784) and Hottest Bone Focal Lesion Deauville score (α = 0.657 and 0.641). Bone Marrow Deauville score showed α values of 0.560 and 0.626, whereas Focal Lesion Group yielded 0.538 and 0.491. Fracture and Extramedullary Disease Presence showed lower agreement (α = 0.280 and 0.235; α = 0.412 and 0.262, respectively). Intra-observer agreement was highest for Hottest Bone Focal Lesion Deauville score (κ = 0.846–0.891), Lytic Lesion Group (κ = 0.759–0.905), and Focal Lesion Group (κ = 0.679–0.855). For exploratory patient-level binary metabolic status, inter-observer agreement was moderate to good (α = 0.611 and 0.542), while intra-observer agreement was good to excellent (κ = 0.626–0.822). No parameter showed a statistically significant change between reading rounds. IMPeTUs supports structured [18F]FDG PET/CT interpretation in multiple myeloma, but reproducibility differs across components, underscoring the need for clearer operational definitions and targeted training for less reproducible parameters.