<b>Empathy-Enhanced Small Language Models for Digital Mental Health Counseling</b>
Access to quality mental health care remains a global challenge, especially in underserved regions. While large language models (LLMs) show promise in providing AI-driven therapeutic support, their massive size often limits practical deployment due to high computational and memory requirements. This study presents a fine-tuned version of Phi-3, a compact yet powerful language model, tailored specifically for empathetic mental health counseling. By leveraging Low-Rank Adaptation (LoRA) and the Unsloth framework, we significantly improved response relevance and emotional sensitivity, while achieving a 2x increase in training speed and reducing memory consumption by 70%. The model was fine-tuned on three diverse dataset of publicly available mental health dialogues, enabling it to generate supportive and context-aware responses. Evaluation using BERTScore metrics demonstrated strong performance, with an F1-score of 0.8429 during training and 0.8385 on unseen test data. Our results suggest that small-scale, efficiently fine tuned LLMs like Phi-3 can offer accessible, accurate, and scalable mental health support bridging the gap between technological capability and real-world usability for the average user.
Authors
- Atif Faridi
- Romana Shehla
Institutions
- Patna University (IN)
- HCL Technologies (India) (IN)
Publication Details
- Journal
- International Journal Of Engineering And Computer Science
- Published
- 2025-08-29
- DOI
- https://doi.org/10.18535/ijecs.v14i08.5218
- Primary Topic
- Digital Mental Health Interventions
- Type
- article
- Field-Weighted Citation Impact
- 0.00