The illusion of algorithmic belonging: why generative AI cannot substitute for relational reciprocity in mental health social inclusion

Purpose This study aims to critically discuss the fast-moving rollout of generative artificial intelligence (AI) and conversational agents as purported remedies to loneliness and social isolation, analysing the fact that the creation of companionship by algorithms substitutes a one-way connection and fundamentally jeopardises real social inclusion. Design/methodology/approach An interdisciplinary perspective that draws from relational sociology, the social model of disability, critical digital studies and various literatures on psychiatric rehabilitation to critique the inter-personal ethics, psychological dangers and political economy of digital mental health companions. It also comprises a comparative analysis of the existing market leaders in conversational AI. Findings Proximate affective containment is provided by generative AI, but lacks of relational reciprocity, moral agency and shared vulnerability. The replacement of civic infrastructures in community settings by automated, conversational agents can set up a divided mental health system: human-centred relational mental health care for the privileged and synthetic, automated containment for the marginalised service user. Moreover, AI companionship takes out user vulnerability as property information, turns it into a commodity, the lived experience of solitude. Research limitations/implications As a perspective, this paper does not contain empirical data of its own; it constructs the argument by conceptual synthesis instead of a systematic review, and the examples of markets discussed (e.g. Woebot, Wysa, Replika) are illustrative, but not comprehensive or evaluative of current versions of products and so may have changed significantly since the time of writing. Longitudinal studies that compare the outcomes of service users using algorithmic companionship with peer-support and Clubhouse models will confirm the synthetic inclusion construct developed here, and could be used for future research. Practical implications Offers evaluative tools to the mental health commissioners, clinicians and third sector organisations to avoid the “community replacement” trap of digital engagement, keeping AI a cognitive prosthetic and not a substitute for interpersonal interactions. Social implications Disciplines policymakers from adopting techno-solutionism as an excuse for austerity-driven disinvestment of physical community centres, peer respites and inclusive civic spaces and calls for the protection of the human social commons. Originality/value This paper outlines the concept of a “synthetic inclusion” that reveals that the illusion of connection created by conversational AI is a commodity, and that social isolation is structurally reinforced. It contributes to the critique of mental health in the digital age by examining unconstrained large language models from a perspective of surveillance capitalism and intersectionality.