Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking

Large language models (LLMs) shape human–AI interaction, yet capability and safety benchmarks provide limited insight into their expressed value priorities. Across three studies, we developed an output-based approach to evaluating correspondence between LLM priorities and human judgements of optimal AI functioning. Study 1 identified six themes through inductive qualitative analysis: Performance, Adaptive Capacity, Social Good, Ethics and Responsibility, Relational Integration, and Agency. Study 2 found stable rankings within models and a shared priority structure across models. Study 3 compared 75 LLMs with 376 human respondents using a profile-fidelity metric combining profile shape and calibration of between-priority differences. Many models reproduced the human profile while exaggerating differences between priorities. Fidelity varied across models, with no clear pattern linking fidelity to more recent releases. Both humans and LLMs assigned Agency low priority. This framework offers a standardized method for auditing expressed value profiles across models alongside capability, safety, and output evaluations.

Authors

Publication Details

Journal
Figshare
Published
2026-11-09
DOI
https://doi.org/10.6084/m9.figshare.34955505
Primary Topic
Ethics and Social Impacts of AI
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking

Andree Hartanto, Fiona Fui‐Hoon Nah, Wei Yan Low, Gabriel Rongyang Lau et al.
Figshare
Ethics and Social Impacts of AI
article

Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking

Andree Hartanto, Fiona Fui‐Hoon Nah, Wei Yan Low, Gabriel Rongyang Lau, Seow Min Koh
article en

Abstract

Large language models (LLMs) shape human–AI interaction, yet capability and safety benchmarks provide limited insight into their expressed value priorities. Across three studies, we developed an output-based approach to evaluating correspondence between LLM priorities and human judgements of optimal AI functioning. Study 1 identified six themes through inductive qualitative analysis: Performance, Adaptive Capacity, Social Good, Ethics and Responsibility, Relational Integration, and Agency. Study 2 found stable rankings within models and a shared priority structure across models. Study 3 compared 75 LLMs with 376 human respondents using a profile-fidelity metric combining profile shape and calibration of between-priority differences. Many models reproduced the human profile while exaggerating differences between priorities. Fidelity varied across models, with no clear pattern linking fidelity to more recent releases. Both humans and LLMs assigned Agency low priority. This framework offers a standardized method for auditing expressed value profiles across models alongside capability, safety, and output evaluations.

Figshare
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon