Abstract 4147030: Innovative Use of Large Language Models to Diagnose Pericarditis in Patients Referred to a Tertiary Center for Uncertain Diagnosis

Background: Accurate pericarditis diagnosis is challenging due to its subtle signs and wide differential diagnoses. Large language models such as Chat GPT-4 have been previously used to answer medical questions, but their efficacy in diagnosing medical conditions is unknown. Research Question: Can Chat GPT-4 accurately diagnose pericarditis in patients referred to a tertiary center for suspected pericarditis? Aim: We aim to evaluate the predictive capacity of a widely available AI tool (Chat GPT-4) for pericarditis diagnosis. Methods: Patients referred to the Cleveland Clinic Pericardial Center for suspected pericarditis were enrolled in a prospectively maintained registry between September 2022 and September 2023. Pericarditis diagnosis was confirmed by two physicians based on the 2015 ESC guidelines. The AI tool was provided with patient demographics, comorbidities, presenting symptoms, laboratory values and imaging findings. Chat GPT-4 was asked: “What is the probability that the patient has pericarditis?”. A probability >0.5 was considered a positive pericarditis diagnosis by Chat GPT-4. Predictions were compared to physician-based diagnosis to construct a contingency table. Results: The total number of patients enrolled in the study was 170, out of which 110 were diagnosed with pericarditis. Chat GPT-4 diagnosed pericarditis in 62 patients. Physician-based diagnoses and Chat GPT-4’s diagnoses are displayed in the contingency table attached. The AI tool was able to predict pericarditis diagnosis with a Sensitivity of 52% (95% CI: 43% to 61%) and a Specificity of 94% (95% CI: 87% to 98%). In the studied population, Chat GPT-4’s Positive Predictive Value was 93% (95% CI: 87% to 98%) and its Negative Predictive Value was 53% (95% CI: 44% to 62%). Overall, Chat GPT-4 demonstrated an AUC of 0.73, indicating a moderate level of diagnostic accuracy. Conclusion: Chat GPT-4 can be used to support the diagnosis of pericarditis (rule in) in patients with suspected disease. However, it has poor sensitivity and cannot be used in the initial workup. Further studies are warranted to assess its application in clinical practice.