Choosing, shedding, setting: three faces of AI responsibility

Abstract Although current AI systems are probably not autonomous agents in a robust traditional sense, we might nonetheless wonder: what capacities must they possess to justifiably be held morally responsible for their choices? This paper argues that the answer to this question depends on the kind of responsibility and choice in question. A system’s ability to choose between programmed goals and means to them may be necessary and (with other capacities) sufficient for the system to justifiably share moral responsibility with its designers, but not for any kind of stand-alone responsibility. By contrast, the ability to shed programmed goals in accordance with moral reasons may be sufficient for holding the system responsible for pursuing a programmed goal rather than not, but not for a contrastive choice between a programmed and a non-programmed goal. To be responsible for such choices, a system must be able to set goals in accordance with moral reasons . I propose operational tests for this capacity and argue that it does not require freedom in an incompatibilist sense; rather, it is analogous to the human capacity to pursue goals not conducive to our survival.