Zhixin: dataset construction and instruction fine-tuning for LLM-based mental disorder diagnosis

Abstract Automatic diagnosis of mental disorder from chief complaints is an appealing but challenging task. Large Language Models (LLMs) have shown rich knowledge and strong reasoning ability in medical domain. However, mental disorder diagnosis based on chief complaints requires highly specialised knowledge. Lack of corresponding resources hinders LLMs performance on this task. To address this problem, we collect 6,780 complaint-diagnosis pairs spanning 29 different types of mental disorders from a specialized hospital as the dataset. Meanwhile we conduct detailed data cleansing and manual proofreading to protect the privacy. On this basis, we propose the model named Zhixin , a LLM for mental disorder diagnosis fine-tuned on the proposed dataset. The experimental results show that Zhixin outperforms the state-of-the-art approaches in terms of automatic evaluation. It surpasses 3-shot DeepSeek-R1 by 18.54 $$\%$$ on Overall Accuracy . Meanwhile, human evaluation results show that the generated response from Zhixin ensures the Accuracy , Safety and Professionalism , which are critical to the diagnosis of mental disorder. In-depth analysis of category-wise performance sheds light on the future research direction, suggesting the focus on relatively rare metal disorders.