An Artificial Intelligence-Based Detection of Comorbid Depression, Anxiety, and Substance Use Disorder in Korean Counseling Dialogues Using an Explainable Hierarchical Attention Network with Shapley Additive Explanations

Background/Objectives: Depression, anxiety disorders, and substance use disorders frequently coexist in clinical settings and are main factors that worsen a patient’s prognosis. However, traditional artificial intelligence-based mental health studies have limitations in capturing the complex symptoms that occur in actual counseling situations by relying on social media data or focusing on binomial classification of single diseases. This study proposes a multi-label classification model that simultaneously detects the coexistence of depression, anxiety, and substance use disorder in actual counseling dialogue texts, and applies the Shapley Additive Explanatory (SHAP) method to explain the clinical basis of model prediction. Methods: We retrospectively analyzed 1661 de-identified Korean-language counseling session transcripts obtained from the publicly available AI Hub “Mental Health Counseling Dialogue” dataset (Republic of Korea; sessions collected between 2021 and 2023 from accredited domestic mental health counseling centers). Each session averaged 30 min (≈5000 Korean characters). Labeling was performed by two licensed clinical psychologists (inter-rater Cohen’s κ = 0.82). A Hierarchical Attention Network with Bidirectional LSTM (HAN-BiLSTM) was constructed; performance was compared with six baselines (Flat LSTM, TextCNN, KR-BERT, KoBERT, KoELECTRA, KLUE-RoBERTa) using stratified 5-fold cross-validation, paired t-tests with Bonferroni correction, and McNemar’s test. Top-ranked SHAP tokens were independently rated for clinical face validity by three psychiatrists. Results: The proposed model outperformed the baseline model not only for the labels of depression (F1 = 0.90) and anxiety (F1 = 0.85) but also for substance use disorder (F1 = 0.78) with poor data, achieving a macro-averaged F1 of 0.84 (95% CI 0.82–0.86; all p < 0.001 versus baselines). As a result of the SHAP analysis, clinically significant keywords such as “I want to die,” “anxiety,” and “drink” were identified as the model’s main basis for judgment, accurately tracking the client’s state, which dynamically changed as the dialogue progressed; three independent psychiatrists rated 88.7% of the top-15 SHAP tokens per label as clinically meaningful (Fleiss’s κ = 0.76). Conclusions: This study demonstrated that a deep learning-based multi-label approach is effective in early screening of complex mental health problems. In particular, the introduction of explainable AI (XAI) increases clinicians’ trust and suggests that it can be used as an AI-based clinical decision support system (CDSS) in the future.