Assessment of online mental health content: integrating lightweight language models with clinical quality criteria

The rapid expansion of online mental health information has created major challenges in sustaining content quality and ensuring user safety and public health outcomes. Manual review processes cannot keep pace with the increasing volume of digital material, and existing automated tools often lack the capacity to deliver comprehensive clinical evaluations. To address these gaps, we introduce an automated domain-specific criterion-based framework that integrates quality assessment criterion for lightweight language models to evaluate mental health content across 20 clinically relevant criteria, including reliability, information safety, cultural sensitivity, and adequacy of crisis-related resources. The models are evaluated on a curated corpus of 416 educational articles from seven authoritative institutional sources. Within this setting, the study focuses on criterion-level quality assessment of institutional educational content and provides a foundation for later extension to commercially biased and adversarial material. Clinical validation shows strong agreement with expert review (89% of high-scoring articles meeting informational standards; model-expert κ = 0.78, n = 100). By enhancing reliability, supporting safer digital health practices, the framework will further support researchers in mental health quality assessment using lightweight language models and has the potential to strengthen the trustworthiness, clinical utility, and public health impact of digital mental health resources.