Response to “Appropriate Statistical Methods to Assess Cross-Study Diagnostic 23-Gene Expression Profile Test Performance for Cutaneous Melanocytic Neoplasms”

To the Editor: We read with great interest the recent publication by Goldberg et al,1 which evaluates the sensitivity and specificity of a non-FDA approved gene expression profiling (GEP) test in categorizing melanocytic lesions as benign or malignant using a standardized statistical approach. However, we are compelled to address a critical methodologic deficiency in their study: the exclusion of indeterminate results from the summary statistics, a practice that deviates from statistical best practices. Although we agree with Goldberg et al that an “apples-to-apples” performance comparison is warranted across studies, one with an appropriate statistical methodology for the clinical scenario is required. The primary role of the GEP test is to help dermatopathologists diagnose challenging lesions that are difficult to classify as benign or malignant based on histomorphology alone. In such cases, a meaningful result from the GEP test is vital for patient treatment. Goldberg et al acknowledge that an indeterminate range was originally implemented “to refine the rate of false-negative test results and provide a buffer between the opposing diagnostic results” for this very purpose. However, when indeterminate results are entirely excluded from the analysis, the reported accuracy metrics of the test are artificially inflated.2,3 This can mislead clinicians about the true diagnostic utility of the test, potentially leading to suboptimal patient care. By considering only definitive positive and negative test outcomes, the analysis fails to consider the complexity of real-world clinical scenarios, where indeterminate results are commonly encountered (8.6%–15.5%).2–4 The FDA emphasizes that indeterminate results should be included in statistical analyses if a test produces outcomes beyond anything other than positive or negative. This is crucial for maintaining transparency and objectivity in diagnostic tests such as the 23-GEP. The FDA warns that ignoring such results “will likely result in biased performance estimates.”5 To address this issue, the FDA suggests reporting 2 sets of performance measures: the first set that combines the equivocal results with the positive results and a second set that combines the equivocal results with the negative results. Furthermore, Shinkins et al6 highlight that excluding indeterminate results is rarely justified and can inflate summary statistics, promoting suboptimal testing strategies. One alternative to excluding valid inconclusive results from binary statistics is to report an additional summary statistic (eg, test yield or effectiveness)—an approach notably absent in Goldberg et al's analysis. However, Shinkins et al also warn against reporting inconclusive results as a separate standalone statistic, because these are often overlooked by readers and may be excluded in meta-analyses, diminishing their impact compared with the primary analysis. Although Goldberg et al adhere to the reporting guidelines outlined by the Standards for Reporting of Diagnostic Accuracy Studies (STARD), their approach to handling indeterminate results is minimally transparent. According to STARD, indeterminate results “can be ignored altogether, be reported but not accounted for, or handled as a separate test result category.” STARD further states that indeterminate results can be reclassified as false positives/negatives (worst-case scenario) or true positives/negatives (best-case scenario) based on the reference standard.7 In our original study,4 we used a refined “worst-case scenario” approach, defining false positives as cases where the histomorphologic diagnosis favored a nevus with a GEP result of either malignant or indeterminate. Similarly, false negatives were defined as cases where the histomorphologic diagnosis favored melanoma with a GEP result that was either benign or indeterminate. We recognize this is a nuanced approach, which was chosen to reflect the clinical scenario of GEP testing in distinguishing between nevus and melanoma and the subsequent decision for excision. We appreciate the concerns raised by Goldberg et al regarding this approach, where indeterminate results could potentially be classified as both false positives and false negatives depending on the histomorphologic diagnosis. To avoid confusion in future studies, we advocate for the more straightforward analysis recommended by Shinkins et al, where “valid inconclusive results are grouped with either the positive or negative results, depending on how these patients would be treated in the particular clinical context,” which is in alignment with the FDA's recommendations. Given the high clinical cost of failing to treat melanoma, inconclusive GEP results should be grouped with positive GEP results (ie, indeterminate result = malignant result), because clinically, these patients should undergo a definitive re-excision. When this approach is applied to the evaluated cohort, the accuracy metrics reported by Goldberg et al change as follows: the sensitivity increases slightly from 92.1% to 93.5%, while specificity reduces from 94.4% to 83.9% (Table 1). TABLE 1. - Updated Accuracy Metrics When Inconclusive GEP Results are Grouped With Positive GEP Results Malignant Histomorphology Benign Histomorphology 23-GEP indeterminate (treated as GEP likely malignant) Malignant indeterminate = 8 Benign indeterminate = 36 23-GEP likely malignant True positive (TP) = 35 + 8 = 43Malignant by histomorphology malignant by 23-GEP False positive (FP) = 16 + 36 = 52Benign by histomorphologyMalignant by 23-GEP 23-GEP likely benign False negative (FN) = 3Malignant by histomorphologyBenign by 23-GEP True negative (TN) = 271Benign by histomorphologyBenign by 23-GEP Accuracy metrics Sensitivity = (TP/(TP + FN) 43/(43 + 3) = 0.935 (93.5%) Specificity = TN/(TN + FP) 271/(271 + 52) = 0.839 (83.9%) Overall indeterminate rate: (8 + 36)/369 = 0.119 (11.9%) Goldberg et al suggest that an indeterminate result could hold value by prompting further diagnostic steps, such as expert consultation or obtaining deeper levels from the biopsy sample. Although we agree these steps are valuable, these should be done before considering costly ancillary testing such as GEP. Expert consultation, additional histologic sectioning, and targeted immunohistochemistry must be exhausted before resorting to an expensive ancillary test. The GEP test is intended to provide clarity when these routine standard-of-care practices fail to yield a definitive diagnosis. If the test fails to provide actionable results, it defeats the very purpose for which it was ordered, ultimately leading to unnecessary expenditures and potentially delaying appropriate patient care. We appreciate the efforts of Goldberg et al in advancing the discussion on the role of GEP testing in dermatopathology, but we believe that the most complete and transparent representation of all possible test outcomes is essential for the test to fulfill its intended role in clinical practice.5 Thus, we encourage Goldberg et al to re-evaluate their outcomes in alignment with the statistical best practices recommended by the FDA and Shinkins et al, where valid inconclusive GEP results are grouped with positive GEP results.