A contextual hierarchical attention network for detecting mental health disorders using social media

Growing parts of the population suffer from mental health problems and psychologists lack capacity to diagnose, let alone treat, all those in need of it. Given recent advancements in the field, deep learning-based NLP techniques could help by detecting those in need of help based on their written text. To this end, this work improves the current state-of-the-art Hierarchical Attention Network (HAN) model by incorporating contextual awareness through BERT-based word embeddings and a multi-head self-attention user-encoder yielding the Context-HAN model. When trained and tested on the eRisk data sets on Self-Harm, Anorexia, and Depression, Context-HAN outperformed the HAN model across all data sets based on various evaluation measures. Furthermore, we find and discuss some interesting insights from analysis of the attention scores, such as that longer and more recently written posts are more important for classification. This work shows the potential of attention mechanisms to leverage contextual information to improve the effectiveness of NLP methods at detecting mental health disorders from user-written text.