Evaluating ChatGPT’s responses on schizophrenia: Accuracy, quality, and stigmatization

Aim: ChatGPT, developed by OpenAI, is an AI-powered language model widely used for online information retrieval.This study aims to assess the accuracy, sufficiency, and stigmatization of ChatGPT's responses on schizophrenia as evaluated by psychiatrists using validated assessment scales.Material and Methods: Common patient queries were identified based on frequently asked questions in clinical practice, relevant healthcare platforms, and previous patient experiences.The questions were formulated with input from ten psychiatry specialists and then posed to ChatGPT in a single session.Psychiatrists evaluated the responses using the Global Quality Scale (GQS), the DISCERN Scale, and a stigma-related question.Results: Among 57 psychiatrists (12 residents, 45 specialists), GQS scores indicated "good to excellent" quality, with higher ratings for treatment-related questions.Responses were generally not perceived as stigmatizing.The DISCERN total score averaged 50, classifying responses as "good," though the lowest ratings were given for source credibility and date information.Discussion: To the best of our knowledge, this is the first study assessing ChatGPT's reliability as an information source on schizophrenia.It provides a valuable reference for future research on the accuracy, reliability, and ethical considerations of evolving large language models in psychiatry.