Beyond human accuracy: application of artificial intelligence for establishing syncope diagnosis in daily practice of the syncope unit

Abstract Introduction and Purpose Artificial intelligence (AI) may transform medicine by improving diagnostic accuracy. History taking is crucial for establishing syncope diagnoses. Diagnostic score tools for the different causes of syncope are not available yet. This study compared AI with the medical professions with regard to the diagnostic yield and accuracy based on patient’s history in the referral letter to the syncope unit. Methods Eighteen medical professions (twelve physicians, six non-physicians) and Azure OpenAI services (GPT-4o) analyzed 55 anonymized referral letters of the general practitioner (GP) and were asked to establish a diagnosis, along with its certainty (graded as certain, highly likely or possible), advice for diagnostic testing and follow up. The gold standard was the diagnosis established in the syncope unit as part from a large volume database. AI performance was tested in a controlled privacy-safe playground environment initiated by the institute. AI was instructed to adhere to the ESC guideline on syncope and practical instruction for which an assist was designed. Statistical analysis was performed by logistic regression. Results The diagnostic yield of de medical professions was 82.8%, the average level of certainty was possible. For GPT-4o this was 100% and highly likely respectively. The diagnostic accuracy of the physicians was 29%, non-physicians 28% and GPT-4o 40%. Epilepsy was diagnosed in 5% by the medical professions and none by GPT-4o. The medical professions suggested additional diagnostic tests such as more comprehensive history in 49.3%, an electrocardiogram in 44%, 24-hour blood pressure (BP) monitoring in 34.5%, holter monitoring in 33.9%, orthostatic BP measurement in 32.5% and tilt table testing in 0.8%. No therapeutical strategies were suggested by the medical professions. GPT-4o, suggested additional diagnostic tests such as tilt table test (81.8%), holter monitoring (56.4%) and orthostatic BP measurement (29.1%). GPT-4o suggested therapeutical strategies such as counter pressure maneuvers (52.7%), increased fluid intake (50.9%), education (45.5%), salt supplementation (40%) and reviewing of medication (32.7%). A statistically significant association was found between a higher certainty of the diagnosis and higher diagnostic accuracy (p=0.032). When GPT-4o was accurate for the diagnosis, there were less additional questions (p=<0.001). Conclusion This study shows that GPT-4o may be of help as a diagnostic tool in the assessment of syncope patients. Medical professions and AI suggested additional diagnostic tests of which most of them could be done by the GP. Only AI suggested therapeutical strategies which could be easily applied even in primary care. This way GPT-4o, may play a role in refining guidelines for the GP which may reduce healthcare costs by reducing the referrals to the syncope unit.Diagnostic Accuracy