Optimizing NLP for Mental Health Analysis: Can DistilBERT Match the Performance of Larger Transformer Models?

Driven by the growing relevance of mental health analysis and the rapid advancement of Transformer-based language models, this study explores the use of DistilBERT, a lightweight Transformer model, for mental health detection based on social media text. Focusing on resource-constrained environments where reliable model performance remains essential, we fine-tuned DistilBERT on publicly available datasets and compared its performance with BERT and RoBERTa using key metrics related to accuracy and computational efficiency. Despite its smaller size, DistilBERT achieved comparable F1-scores while significantly reducing inference time and memory usage. The comparative experiments demonstrate that DistilBERT offers a practical balance between accuracy and efficiency, highlight its potential as an efficient and scalable solution for mental health monitoring systems.