An Amalgamation of RNN and Transformers for Identifying Depression at the Preliminary Stage

The emergence of social media platforms has facilitated the effortless dissemination of individuals’ thoughts, emotions, concerns, and accomplishments to a global audience. The utilization of deep learning techniques to analyze social media data for early identification of depression has the potential to drastically transform the field of depression detection since a significant number of individuals express their emotions through social media platforms. This paper proposes a novel method for detecting depression at an early stage using transformer-based models. The data utilized for the experimentations is the publicly available dataset from Kaggle, consisting of 10,314 tweets. The tweets are pre-processed to clean the text, and then tokens are generated using the transformer-based models. The generated tokens are provided as input to the proposed transformer-based models like bidirectional encoder representations from Transformer (BERT), the robustly optimized BERT approach (Roberta), and the hybrid model, which combines BERT with Recurrent neural networks (RNN) and Roberta with RNN. The embeddings are extracted using the pre-trained models, and classification layers for the initial layers are added. Among the four models, it is observed that the hybrid models perform well by achieving a validation accuracy of 99.9%. This study adds to the expanding area of user-generated content analysis by providing essential visions into the possibility of early depression detection and monitoring through Internet platforms.