When Human Values Become Rewards

Choosing a reward for an AI system is also a moral choice because it ranks responses and shapes learning. Successful training may show that the system follows this ranking, but not that it is justified or should govern every case. This article uses normative-conceptual analysis to examine how ethical frameworks become AI objectives and which moral judgments that conversion requires. It compares consequentialist, Kantian, virtue-based and contractualist approaches, then uses Theravāda Abhidhamma as a hard case because its fine-grained ethical structure makes the translation problem especially visible. The analysis identifies seven bridge judgments concerning what is evaluated, how ethical categories apply to a machine, what evidence supports that application, how considerations are compared, where the objective applies, whose interpretation governs and when revision is required. Because no ethical source settles all seven, technical success cannot make a reward morally final. The article develops reward finalism and the moral finality gap, derives a principle of reward non-finality, and proposes a seven-question audit for making the judgments behind alignment claims explicit and treating rewards in open-ended settings as non-final specifications open to challenge and revision.