Performance Comparison of Word Embedding and Sequential Models for Mental Health Classification

Mental health classification from text has made significant progress in recent years, following advances in distributional and contextual word representations, combined with sequential models. In this paper, we compare the previous approaches which integrate word embeddings (e.g., Word2Vec, GloVe, FastText), and language models (e.g., BERT/ClinicalBERT) of sequence architectures Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), Gated Recurrent Unit (GRU), CNN-LSTM, Transformers into detection of depression and its comorbidities such as stress and anxiety. We review and discuss datasets, preprocessing methodologies, model settings and evaluation protocols, as well as, we compile reported results into common comparison tables and visual overviews. The study states clear benefits provided by contextual embeddings and bidirectionality in sequence modeling but also poses several issues related to dataset mismatch of domains, class imbalance, interpretability, as well as cross-domain generalization. We close with directions for model selection within practical constraints and discuss open problems towards clinically reliable deployment.