Is Meaningful Human Control Over Personalised AI Assistants Possible? Ethical Design Requirements for The New Generation of Artificially Intelligent Agents

Abstract Recently, several large tech companies have pushed the notion of AI assistants into the public debate. These envisioned agents are intended to far outshine current systems, as they are intended to be able to manage our affairs as if they are personal assistants. In turn, this ought to give users a leg up, as one prominent tech exec has put it. However, it remains to be seen how these Personal AI Assistants (PAIAs) are implemented, and critical reflection on how and whether they can be implemented in a responsible way is needed. Currently, such agents are undertheorized and this may cause us to misunderstand their value and capacity. In this paper, we explore and critique the potential for responsible implementation by considering some design requirements based on the notion of meaningful human control. If we desire to have control over such assistants, then we need to be able to do so meaningfully and effectively. In looking at the design requirements, we run into the issue that their broad and differing capacities make any kind of design requirements hard because there are simply no standards to which we can measure PAIAs. Furthermore, it seems that the implementation of these assistants will be a matter of trade-offs both in capacities and in values, which will likely lead to enhancement for some rather than an improvement for all.