Living with AI-extenders: ethical challenges to mental autonomy

Abstract The hypothesis of extended cognition has reshaped our understanding of human mind as fluid and without fixed boundaries: certain suitably integrated technologies can become proper parts of the cognitive machinery that realises our thinking, memory, and problem-solving abilities. This paper zooms in on how the extension relation changes, when instead of a passive tool like a notebook the extension is actualised with AI technology. “AI-extenders” learn from each user’s habits and needs, and they adapt their responses over time. This process allows AI-extenders to integrate directly into daily thinking routines. Besides everyday use, they can be used for therapeutic purposes, such as for people suffering from Alzheimer’s. At the same time, they may undermine our mental autonomy through subtle manipulation. Even if AI-extenders meet the criteria for genuine cognitive extension, this ontological claim doesn’t settle the normative question of what we ought to do about it. The ethical risks they pose may give us reason to take practical measures to decouple the extended system.