Uncanny Human Kinds: the Problematic Social Life of Deep Learning-Generated Health Categories

Abstract There is marked enthusiasm about the potential for deep learning (DL) systems to generate new data-driven diagnostic, predictive, and prognostic health categories that could support more effective, precisely targeted healthcare. This article argues that, in order to determine whether these novel categories would be ethically desirable ones, we cannot look only to their clinical utility, or to the threats posed to this by widely recognised problems of algorithmic error, bias, and opacity. This is because health categories do not only classify disease and inform care. They also classify people. And they do so for ethically impactful purposes that extend far beyond the clinic. In light of this, and to address the central question of whether novel DL-generated health categories would constitute good or supportive ways of classifying people, this article appeals to the concept of human kinds. This framing serves to highlight how diagnostic and predictive health categories can shape the identities and relationships of the people classified in ethically significant ways. It is argued here that DL-generated health categories could occupy the roles of human kinds. However, they are likely to exhibit particular qualities that distinguish them from familiar human kinds that have originated in human practices. And these qualities could result in marked disjunctions between, on one hand, the meanings and membership of these kinds, and on the other hand, members’ characteristics and lived contexts, resulting in uncannily abstracted kinds. As a result of this uncanniness, kinds originating in DL-generated health categories could significantly undermine the wellbeing and interests of their members in a range of important ways. For this reason, when assessing the ethical desirability of using DL to generate data-driven diagnoses and health predictions, it is necessary to assess not only the clinical value of these diagnoses and predictions, but also the risks of seeding uncanny human kinds.