21 Optimizing AI-physician collaboration for enhanced diagnostic accuracy: A case study on acute respiratory distress syndrome detection using chest X-ray imaging

Objectives/Goals: The objective of this study is to explore strategies for AI-physician collaboration in diagnosing acute respiratory distress syndrome (ARDS) using chest X-rays. By comparing the diagnostic accuracy of different AI deployment methods, the study aims to identify optimal strategies that leverage both AI and physician expertise to improve outcomes. Methods/Study Population: The study analyzed 414 frontal chest X-rays from 115 patients hospitalized between August 15 and October 2, 2017, at the University of Michigan. Each X-ray was reviewed by six physicians for ARDS presence and diagnostic confidence. We developed a deep learning AI model for detecting ARDS and explored the strengths, weaknesses, and blind spots of both physicians and AI systems to inform optimal system deployment. We then investigated several AI-physician collaboration strategies, including: 1) AI-aided physician: physicians interpret chest X-rays first and defer to the AI model if uncertain, 2) physician-aided AI: the AI model interprets chest X-rays first and defers to a physician if uncertain, and 3) AI model and physician interpreting chest X-rays separately and then averaging their interpretations. Results/Anticipated Results: While the AI model (84.7% accuracy) had higher accuracy than physicians (80.8%), we found evidence that AI and physician expertise are complementary. When physicians lacked confidence in a chest X-ray’s interpretation, the AI model had higher accuracy. Conversely, in cases of AI uncertainty, physicians were more accurate. The AI excelled with easier cases, while physicians were better with difficult cases, defined as those where at least two physicians disagreed with the majority label. Collaboration strategies tested include AI-aided physician (82.4%), physician-aided AI (86.9%), and averaging interpretations (86%). The physician-aided AI approach had the highest accuracy, could off-load the human expert workload on the reading of up to 79% chest X-rays, allowing physicians to focus on challenging cases. Discussion/Significance of Impact: This study shows AI and physicians complement each other in ARDS diagnosis, improving accuracy when combined. A physician-aided AI strategy, where the AI defers to physicians when uncertain, proved most effective. Implementing AI-physician collaborations in clinical settings could enhance ARDS care, especially in low-resource environments.