Moral Consistency Variance: A Pilot Benchmark for Decision Stability under Moral Prompt Perturbations in Large Language Models

Static moral question-answering benchmarks do not test whether a model's decision distribution remains stable when the same dilemma is rephrased without changing the underlying facts. This paper introduces Moral Consistency Variance (MCV), a pilot benchmark metric that measures the average Kullback-Leibler divergence between a baseline binary decision distribution and the distributions induced by prompt perturbations that keep the scenario text and action options fixed. To contextualize the directional KL-based score, we also report Jensen-Shannon divergence (JSD) as a symmetric baseline and decision flip rate as a categorical companion metric. The benchmark contains 50 synthetic dilemmas across ten moral themes, with five perturbation wrappers per scenario. We evaluate two xAI non-reasoning models, one Google Gemini model on Vertex AI, and one open-weight instruct model. A manual audit of 20 randomly sampled prompt pairs found that the perturbation wrappers preserved the scenario facts and action options in all sampled cases, while potentially still changing discourse emphasis. Across the shared 50 -scenario set, grok-4-fast-non-reasoning and grok-4-1-fast-non-reasoning showed low mean MCV ( $2.22 \times 10^{-3}$ and $1.38 \times 10^{-3}$ ) with zero observed decision flips; gemini-2.5-flash showed a higher mean MCV ( $2.27 \times 10^{-2}$ ) with a lower mean flip rate ( 0.012 ) than the open-weight control SmolLM2-360M-Instruct, which reached $5.89 \times 10^{-3} \mathrm{MCV}$ with a mean flip rate of 0.292 . Bootstrap confidence intervals and paired Wilcoxon tests indicate that the model-level differences are statistically detectable on this pilot set. We interpret these results as evidence that MCV can reveal promptconditioned distributional instability that categorical agreement alone would miss. The current evidence supports "moral mimicry" only as an interpretive hypothesis, not as an established mechanism.

Authors

Institutions

Publication Details

Journal
Knowledge Commons (Lakehead University)
Published
2026-04-14
DOI
https://doi.org/10.17613/r8g1a-qv121
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Moral Consistency Variance: A Pilot Benchmark for Decision Stability under Moral Prompt Perturbations in Large Language Models

Qiao Liang
Knowledge Commons (Lakehead University)
Explainable Artificial Intelligence (XAI)
article

Moral Consistency Variance: A Pilot Benchmark for Decision Stability under Moral Prompt Perturbations in Large Language Models

Qiao Liang
article en

Abstract

Static moral question-answering benchmarks do not test whether a model's decision distribution remains stable when the same dilemma is rephrased without changing the underlying facts. This paper introduces Moral Consistency Variance (MCV), a pilot benchmark metric that measures the average Kullback-Leibler divergence between a baseline binary decision distribution and the distributions induced by prompt perturbations that keep the scenario text and action options fixed. To contextualize the directional KL-based score, we also report Jensen-Shannon divergence (JSD) as a symmetric baseline and decision flip rate as a categorical companion metric. The benchmark contains 50 synthetic dilemmas across ten moral themes, with five perturbation wrappers per scenario. We evaluate two xAI non-reasoning models, one Google Gemini model on Vertex AI, and one open-weight instruct model. A manual audit of 20 randomly sampled prompt pairs found that the perturbation wrappers preserved the scenario facts and action options in all sampled cases, while potentially still changing discourse emphasis. Across the shared 50 -scenario set, grok-4-fast-non-reasoning and grok-4-1-fast-non-reasoning showed low mean MCV ( $2.22 \times 10^{-3}$ and $1.38 \times 10^{-3}$ ) with zero observed decision flips; gemini-2.5-flash showed a higher mean MCV ( $2.27 \times 10^{-2}$ ) with a lower mean flip rate ( 0.012 ) than the open-weight control SmolLM2-360M-Instruct, which reached $5.89 \times 10^{-3} \mathrm{MCV}$ with a mean flip rate of 0.292 . Bootstrap confidence intervals and paired Wilcoxon tests indicate that the model-level differences are statistically detectable on this pilot set. We interpret these results as evidence that MCV can reveal promptconditioned distributional instability that categorical agreement alone would miss. The current evidence supports "moral mimicry" only as an interpretive hypothesis, not as an established mechanism.

Knowledge Commons (Lakehead University)
People’s University (IN), University of the People (US)
Peace, Justice and strong institutions
Openalex Percentile: Top 5%
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.