A Study of the Development and Deployment of Emotionally Intelligent Artificial Intelligence

This paper examines the relationship between artificial intelligence (AI) and emotional intelligence (EI), particularly in light of the growing deployment of emotionally responsive AI systems in sensitive human contexts (e.g., healthcare, education, and customer service). Although AI has made significant progress in understanding and simulating human emotions through affective computing, it remains structurally incapable of experiencing emotions, as AI lacks consciousness and self-awareness—a phenomenon known as the "empathy gap." At an emotional level, AI can simulate empathy through data-driven models. Emotionally intelligent AI can approximate empathy through its data-driven model, but these simulations are not equivalent to genuine empathy. This distinction also raises ethical concerns, as users may experience emotional reactions or develop trust in AI systems that can't actually comprehend or grasp emotions. Such an illusion of empathy, enforced by HAT for humans to harmonically anthropomorphise machines, can make some individuals be emotionally dependent and/or manipulated (especially the ones who are vulnerable). It also highlights key risks associated with emotionally intelligent AI, including bias in emotion recognition, emotional manipulation by both commercial and political actors, and the disruption of genuine human interactions. It acknowledges that although AI can augment users' enjoyment and engagement, it must be implemented and designed ethically and transparently if it is to remain a supportive tool rather than a replacement for genuine human empathy.