Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking
Large language models (LLMs) shape human–AI interaction, yet capability and safety benchmarks provide limited insight into their expressed value priorities. Across three studies, we developed an output-based approach to evaluating correspondence between LLM priorities and human judgements of optimal AI functioning. Study 1 identified six themes through inductive qualitative analysis: Performance, Adaptive Capacity, Social Good, Ethics and Responsibility, Relational Integration, and Agency. Study 2 found stable rankings within models and a shared priority structure across models. Study 3 compared 75 LLMs with 376 human respondents using a profile-fidelity metric combining profile shape and calibration of between-priority differences. Many models reproduced the human profile while exaggerating differences between priorities. Fidelity varied across models, with no clear pattern linking fidelity to more recent releases. Both humans and LLMs assigned Agency low priority. This framework offers a standardized method for auditing expressed value profiles across models alongside capability, safety, and output evaluations.
Authors
- Andree Hartanto (ORCID: https://orcid.org/0000-0001-8758-6400)
- Fiona Fui‐Hoon Nah (ORCID: https://orcid.org/0000-0002-5505-7843)
- Wei Yan Low (ORCID: https://orcid.org/0009-0004-0031-8577)
- Gabriel Rongyang Lau (ORCID: https://orcid.org/0009-0001-7926-8266)
- Seow Min Koh
Publication Details
- Journal
- Figshare
- Published
- 2026-11-09
- DOI
- https://doi.org/10.6084/m9.figshare.34955505
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- article
- Field-Weighted Citation Impact
- 0.00