#3548 ChatGPT put to the test: diagnosis and treatment of glomerular diseases in the real world—a pilot study

Abstract Background and Aims Artificial intelligence (AI) is increasingly becoming integral to modern medicine, with applications ranging from diagnostics to treatment recommendations. Among the advancements, large language models like ChatGPT have shown the potential to aid clinical decision-making by synthesising vast amounts of medical knowledge. However, these models' actual utility and limitations in real-world clinical scenarios remain underexplored, particularly in diagnosing and managing complex diseases such as glomerular disorders. Our pilot study aims to evaluate ChatGPT's ability to simulate nephrological expertise in diagnosing and formulating treatment strategies for glomerular diseases. Method We retrospectively selected anonymised data from 10 randomly chosen patients who underwent diagnostic kidney biopsy between November 2022 and June 2023 at the University Medical Centre Maribor, with at least one year of clinical follow-up available. Comprehensive clinical information, including patient history, physical examination findings, and laboratory results, was compiled and used by pathologists to support clinicopathological correlation. This dataset was then input into ChatGPT, where the model was prompted to provide the three most likely diagnoses for each patient's condition. Subsequently, anonymised histopathology reports were uploaded, and ChatGPT was asked to recommend a treatment plan and estimate the likelihood of disease remission at one year. Results The median age of the patients was 53.5 years (IQR 15), with 50.0% being female. Histopathological diagnoses included membranous nephropathy (n = 2), focal segmental glomerulosclerosis (n = 2), IgA nephropathy (n = 2), minimal change disease (n = 1), MPO-ANCA vasculitis (n = 1), combined MPO-ANCA vasculitis and IgA nephropathy (n = 1), and immune complex glomerulonephritis (n = 1). ChatGPT correctly identified the exact diagnosis based on patient history and laboratory data in 4 of 10 cases and included the correct diagnosis among its top three differential diagnoses in 7/10. After adding histopathology data, ChatGPT's treatment recommendations aligned with the treating physician's approach in 6 of 10 cases. ChatGPT estimated a one-year remission probability of 70.0%. At one-year follow-up, 6/10 of patients achieved complete remission, two patients achieved partial remission, and two patients showed no response to treatment. None of the patients died or required dialysis. Interestingly, all patients for whom ChatGPT's treatment recommendations differed from those of the treating physician achieved complete remission. Conclusion ChatGPT demonstrated promising potential in diagnosing and managing glomerular diseases, correctly identifying the diagnosis and aligning treatment recommendations with physicians in many cases. Its predictions for one-year remission were consistent with actual outcomes, highlighting its utility as a supportive tool in nephrology. Further studies are needed to refine its application in clinical practice.