Who Punishes LLM Alignment? User Characteristics and the Evaluation of Ethical Response Refusals
Ethical refusals are essential for aligning large language models (LLMs) with safety constraints, yet users often penalize them in comparative evaluations. Whether this refusal penalty varies systematically across users remains unclear. Drawing on human–computer interaction research, we examine how users’ evaluative priorities and usage intensity shape responses to ethical refusals. Using the PRISM alignment dataset, which links LLM evaluations to user characteristics, we find that safety preferences significantly mitigate refusal penalties, whereas preferences for helpfulness, fluency, and creativity do not. Heavy users penalize refusals more strongly, an association substantially mediated by lower safety preferences. These findings suggest that acceptance of safety-aligned behavior depends primarily on users’ safety orientation rather than preferences for instrumental response qualities. Our results highlight the limitations of evaluating LLM safety alignment through aggregate user preferences and underscore the importance of accounting for user heterogeneity in evaluation benchmarks and deployment decisions.
Authors
- Stefan Pasch (ORCID: https://orcid.org/0000-0002-0197-0442)
- Min Chul Cha (ORCID: https://orcid.org/0000-0001-9301-4281)
Publication Details
- Journal
- Figshare
- Published
- 2026-11-18
- DOI
- https://doi.org/10.6084/m9.figshare.35048826
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- article
- Field-Weighted Citation Impact
- 0.00