The Role of Empathy in Critical Reasoning and the Limitations of Medical AI Systems

The recent developments of medical AI systems (MAIS) open up questions as to whether and to what extent MAIS can be modeled to include empathetic understanding, as well as what impact MAIS' lack of empathetic understanding would have on its ability to perform the necessary critical analyses for reaching a diagnosis and recommending medical treatment. In this article, we argue that current medical AI systems' ability to empathize with patients is severely limited due to its lack of first-person experiences with human interests and that efforts to correct for this deficit-by having MAIS interpret patients' medical and non-medical interests-will encounter significant obstacles. Finally, we demonstrate how MAIS' lack of empathy is likely to hinder its performance in crucial aspects of the processes through which useful medical diagnoses are reached and through which appropriate treatment recommendations for patients are determined.