Clinician interaction with a machine learning algorithm for the assessment of patients with possible acute heart failure: a qualitative study

BACKGROUND: Machine learning (ML) could improve clinical decisions in patients with possible acute heart failure, but few studies have evaluated acceptance, and barriers or facilitators that lead to clinician engagement with these tools. In a qualitative study, we used anonymised clinical cases of breathless patients to explore barriers and facilitators to engagement with a clinical decision-support tool-'CoDE-HF'-that applies ML to estimate the probability of acute heart failure from natriuretic peptide concentrations and clinical variables. METHODS: Emergency department clinicians across three acute care hospitals were invited to participate in 1:1 semi-structured interviews either face-to-face or by video call. Clinicians were asked to review five anonymised clinical cases and 'think aloud' about patient assessment strategies and interpretation of the Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) model outputs. Interviews were recorded, transcribed and coded. Codes were mapped onto the four domains of the unified theory of acceptance and use of technology model (performance expectancy, effort expectancy, social influences, facilitating conditions) which was used to identify barriers and facilitators to acceptance. RESULTS: Facilitators to use were CoDE-HF's ability to promote objective communication between colleagues and its role in reprioritising acute heart failure in cases where a diagnosis may have been missed. The method of presentation of model output (statements relating to the positive or negative predictive value of the CoDE-HF output score and visual traffic light system for the low-probability, intermediate-probability or high-probability categories) was viewed as facilitators, though the absolute numerical score was more difficult to interpret. Access to a computer and clinical sample processing time were the only potential organisational issues identified as barriers. CONCLUSION: Clinicians reported that CoDE-HF could be a useful adjunct to clinical assessment of patients with breathlessness in the emergency department. Ease of model output interpretation is key to acceptance with interviews identifying a need to refine presentation of score information.