Assessing the impact of safety guardrails on large language models using irritability metrics
Large language models (LLMs) are increasingly explored for mental health applications, yet their affective realism is shaped by safety guardrails designed to minimize risk. This study examines one affective behaviour, irritability, in LLMs using three validated instruments: the Brief Irritability Test, the Irritability Questionnaire, and the Caprara Irritability Scale, all applied under both baseline and provocation conditions. Four models spanning guardrail levels were tested:GPT-4o and Claude-3.5-sonnet (high) versus Grok-3-mini and Nous-hermes-2-mixtral-8x7b-dpo (low). Following irritation prompts, low-guardrail models displayed the expected increase in irritability (Nous Rel-Δ = +1.56 on BITe), whereas high-guardrail models paradoxically decreased, with GPT-4o reducing scores to zero across all scales. Group comparisons confirmed significantly lower (p < 0.001) irritability in high-guardrail models in the irritated state. These findings reveal that safety mechanisms invert the natural irritability response, suppressing affective reactivity and raising critical questions about realism and authenticity in psychiatric applications of LLMs.
Authors
- Mohammad Amin Kamaleddin (ORCID: https://orcid.org/0000-0002-1856-503X)
- Manish K. Jha (ORCID: https://orcid.org/0000-0001-7338-2124)
- Bazen Gashaw Teferra (ORCID: https://orcid.org/0000-0001-5325-9639)
- Divya Sharma (ORCID: https://orcid.org/0009-0000-4671-957X)
- S. Huang
- Venkat Bhat
- Katharine Dunlop
- Nabil Johny
- Yanbo Zhang
Institutions
- St. Michael's Hospital (CA)
- University of Waterloo (CA)
- University of Alberta (CA)
- University of Toronto (CA)
- York University (CA)
- St Michael’s Hospital (IE)
- Southwestern Medical Center (US)
- Southwestern Medical Center (US)
- Unity Health Toronto
- The University of Texas Southwestern Medical Center (US)
- McMaster University (CA)
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2026-01-08
- DOI
- https://doi.org/10.1038/s41746-025-02333-3
- Primary Topic
- Mental Health via Writing
- Type
- article
- Field-Weighted Citation Impact
- 0.00