Evaluation of Lung Cancer Probability Models and Guideline Recommendations in Settings With a High Prevalence of Cancer

<h3>Background</h3> Guideline-recommended management of patients with pulmonary nodules includes assessing the probability of cancer before testing. The performance of 4 models (the Mayo, Brock, Veterans Affairs [VA], and Peking University [PKU] models) has been validated in several populations, but not in a high-prevalence setting. <h3>Research Question</h3> What is the performance of lung cancer probability models within the context of a population with a high prevalence of lung cancer at a tertiary care center? <h3>Study Design and Methods</h3> Clinical and radiologic data were reviewed retrospectively for 1,518 patients with 6- to 30-mm nodules referred to a pulmonologist or thoracic surgeon at Vanderbilt University Medical Center and Veterans Affairs Tennessee Valley Healthcare System Nashville Campus from 2002 through 2021 for evaluation and treatment of pulmonary nodules. Probability of cancer for each patient was calculated according to 4 validated models. Model performance was assessed based on receiver operating characteristic (ROC), calibration, sensitivity, and specificity. <h3>Results</h3> Of the total cohort (N = 1,518), 1,098 patients (72.3%) harbored a malignant nodule. The Mayo model discriminated between patients with benign and malignant nodules with the highest area under the ROC curve (AUC), 0.74. The Brock and VA models performed similarly in cohort discrimination (AUC, 0.71 and 0.70, respectively), but the VA model was better calibrated (Brier score, 0.24). The PKU model discriminated between benign and malignant nodules with the lowest AUC (0.66), but the best calibration (Brier score, 0.19). <h3>Interpretation</h3> The classification accuracy of the models did not differ greatly, but the calibration, and resulting sensitivity and specificity at guideline-recommended thresholds, was greatly dependent on the prevalence of cancer in the population in which the model was trained. The prevalence of cancer in the clinical setting should be considered when using a clinical prediction model for indeterminate pulmonary nodule management.