Autonomous and AI-enabled systems: extensions or replacements of human will and control?

Abstract Use of autonomous, AI-enabled, or opaque systems raises many concerns, and some argue that for these to be permissibly deployed in high-stakes or critical domains, they must be subject to so-called “meaningful human control” (MHC). In this article, I focus on the military domain and rebut a strong version of this critique, arguing that off-the-loop systems – i.e., those which can select and engage targets without contemporaneous human input or oversight – can be permissibly deployed while retaining clear lines of responsibility and control. I show that ex ante operational constraints and targeting parameters can provide combatants and would-be deployers of off-the-loop systems with strong means to ensure that deployed systems are serving as extensions of combatants’ wills, establishing the necessary degree of moral and legal responsibility required. I further show that such constraints and parameters represent clear lines of control that deployers have over such systems, even when these systems, during deployments, are utterly outside of human control. I conclude by distinguishing between what I call “will-extending” and “will-offloading” systems, showing that off-the-loop systems can serve to extend users’ and deployers’ wills, making such systems inherently subject to meaningful human control. Throughout, the discussion focuses on the example of autonomous and AI-enabled systems in the military domain, but the underlying arguments relate to such systems more generally, showing how these, if utilized as “will-extending” systems, may be used in a controlled and responsibility-retaining manner.