Evaluating ChatGPT's Diagnostic Capabilities for Mental Health Disorders

The field of artificial intelligence (AI) has seen significant advancements in recent years, making it a notable technological achievement in various aspects of daily life. In this study, we sought to investigate the feasibility of employing AI in the realm of mental health. Specifically, we assessed the efficacy of ChatGPT as a diagnostic tool for mental health disorders. To this end, 25 vignettes depicting common mental disorders were presented to ChatGPT, and its diagnostic accuracy was evaluated across three experimental conditions (the original vignette, the vignette with gender switch, and a shortened version of the vignette). The results showed high accuracy rate, surpassing random guessing, and highlighted ChatGPT's adherence to specific diagnostic criteria. This accuracy persisted even when the vignettes depicted rare mental disorders. These findings are discussed with an emphasis on potential gender biases, the risks tied to self-diagnosis, and the pressing need for further validation and ethical considerations. The study concludes by addressing the potential for incorporating ChatGPT into the broader realm of mental health in the future.