Abstract 4142075: From GPT-4 to GPT-4o: Progress and Challenges in ECG Interpretation

Background: With the increasing popularity of large language models (LLM), the use of artificial intelligence (AI) for medical diagnostics has garnered significant attention. One area of particular interest is the application of LLMs in the interpretation of electrocardiograms (ECGs). Traditional ECG machines, such as those by General Electric (GE), use established algorithms for analysis. This study explores the potential of AI, specifically OpenAI’s GPT-4 and its latest iteration, GPT-4o, as an aide in interpreting ECGs. Objective: To evaluate the improved capability of GPT-4o in interpreting ECGs, comparing its performance against GE’s algorithm and previous iteration GPT-4, with a focus on identifying both improvements and remaining limitations. Methods: A total of 25 randomly selected ECGs representing both normal and abnormal findings were analyzed using GPT-4o. We prompted GPT-4o to comment on clinically relevant ECG findings, including rate, rhythm, QRS axis, intervals, chamber enlargement, and signs of ischemia. We compared the interpretations of GPT-4o against the GE machine and an older version of ChatGPT, examining diagnostic accuracy, consistency, and reliability. Results: GPT-4o showed marked improvements in detecting critical diagnoses compared to older versions. It successfully identified 4 STEMIs and all ventricular tachycardias (VT) missed by its predecessor. However, GPT-4o continued to struggle with atrial arrhythmias, missing 2 cases of atrial fibrillation, 1 atrial flutter, and 1 supraventricular tachycardia. Interestingly, GPT-4o exhibited a higher tendency to classify nonspecific ST segment and T wave changes as ischemia. Notable limitations included the quality of ECGs, with some having poor baselines, and the resolution of uploaded images. Conclusion: The rapid improvement in ECG interpretation, especially in detecting critical diagnoses previously missed, is very promising for AI and ChatCPT. GPT-4o shows significant advancements over its predecessor, particularly in identifying STEMIs and VTs. However, it still struggles with atrial arrhythmias and tends to overdiagnose ischemic changes, indicating the need for further refinement. While AI advancements are promising, more development is necessary before such technologies can be safely integrated into routine ECG interpretation. Future research should focus on enhancing AI's ability to detect a broader range of arrhythmias and improving efficiency in cardiology diagnostics.