S920 Diagnostic Codes Performance for Identification of Eosinophilic Gastrointestinal Diseases Varies by Reference Standard Definition and Disease Subtype

Introduction: Eosinophilic gastrointestinal diseases (EGIDs) are understudied. Internal Classification of Diseases (ICD) codes are a potential tool to facilitate EGID research, but their accuracy is unknown. We aimed to evaluate the ICD performance metrics for identifying EGIDs using multiple case definitions and across subtypes. Methods: We identified GI pathology reports containing “eosinophil” and paired them with preceding clinic notes from a large health system. Reference standard diagnoses were assigned via chart review and compared to ICD codes. The EoE reference standard required positive histology with variable definitions of esophageal dysfunction and diagnostic history. Defining reference standard for eosinophilic gastritis (EoG) and eosinophilic enteritis (EoN) included liberal (any eosinophils plus symptoms) and restrictive (increased eosinophils plus symptoms) criteria; only liberal criteria were used for eosinophilic colitis (EoC) due to limited detail. For non-EoE EGIDs, a rule-based natural language processing approach captured numeric or descriptive eosinophil increases. EoG and EoN, lacking distinct ICD codes, were evaluated individually and combined under the eosinophilic gastroenteritis code. Performance metrics included sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, Cohen’s kappa, and F1 score (harmonic mean of precision and recall). Results: Defining EoE reference standard with symptoms of dysphagia and food impaction only, the ICD achieved an F1 score of 0.70 (95% confidence interval [CI] 0.59-0.80), with a PPV of 0.64 and sensitivity of 0.77. Including additional esophageal dysfunction symptoms yielded a modest improvement in F1 score (ΔF1 = 0.02; 95% CI 0.0 to 0.08). Further addition of history of EoE documented in note free text resulted in higher PPV (0.95), specificity (0.98), and F1 score (0.84; 95% CI 0.78-0.90), with similar sensitivity (0.76). The diagnostic performance of ICD codes for identifying EoG or EoN demonstrated low PPV (0.17-0.33) and sensitivity (0.08-0.33) across individual subtypes and in combination, with slightly better performance for restrictive definitions. No patients with EoC received the corresponding ICD code, resulting in zero sensitivity and undefined PPV. Conclusion: The diagnostic performance of ICD codes for EoE is influenced by the specific criteria and clinical data elements incorporated in the reference standard definition. The ICD codes for non-EoE EGIDs demonstrate high false positive and false negative rates. These findings highlight the need for improved case identification methods to facilitate EGID research.