Abstract 4146609: Evaluating the performance and potential bias of predictive models for the detection of transthyretin cardiac amyloidosis

Background: Transthyretin amyloid cardiomyopathy (ATTR-CM) is a highly morbid cause of heart failure (HF). Early detection enables initiation of novel therapies to halt progression and drastically improve outcomes. Current evidence suggests that Black communities are disproportionately impacted by underdiagnosis. Models developed to detect ATTR-CM have been proposed as a screening strategy to address this unmet need, but trust in model outputs and concern for bias remain a barrier to adoption. We externally validated the performance of predictive screening tools and audited the models for bias on a retrospective cohort across a large health system. Methods: We identified patients treated at an integrated health system from 2010-2022 with biopsy or PYP scan-confirmed ATTR-CM and age -and sex-matched them to controls with HF in a 19:1 ratio to target 5% prevalence. We then compared the performance of three publicly available algorithms: a random forest model of claims data, the regression-based Mayo ATTR-CM score, and a deep-learning echo model (EchoNet-LVH). Bias was measured in the best performing models using standard fairness metrics. Results: We identified 245 confirmed cases of ATTR-CM for 4900 patients total in our analytic cohort. We excluded 892 patients (41 cases) in which EchoNet-LVH could not be run due to image quality or other factors. In the remaining 4008 patients, 78.7% of the cohort self-identified as White, 8.8% Black, 4.0% Hispanic and 8.5% Other. ATTR-CM prevalence was highest in individuals who identified as Black. The claims-based model performed poorly with an AUC of 0.48. EchoNet-LVH had higher AUC (0.88 vs 0.78) and average precision (0.62 vs 0.18) compared to the Mayo score. Results from bias auditing of the top performing models are presented in the Table below. Conclusions: In external validation using a large, diverse cohort of patients with heart failure, a deep-learning echo-based model to detect ATTR-CM demonstrated best overall performance compared to two other publicly available models. The results of a bias audit suggest that the model is unlikely to exacerbate existing health disparities through inequitable distribution of error with respect to self-identified Black race.