WHAT ADOLESCENTS SEEK IN ARTIFICIAL COMPANIONS AND WHAT STATES SHOULD DO ABOUT IT

Adolescents are telling conversational systems what they tell no one else. They do so in large numbers, often late at night, and most often without mentioning it to anyone. The practice took hold in three years, faster than any child-protection framework has been able to follow, and the first regulations written specifically for these systems have only just entered into force.This policy brief asks why adolescents turn to these systems, what is known about the harms and the benefits, whether the applications that present themselves as psychological support deliver what they promise, how the resulting data is handled, and what states have begun to do. It argues that the need driving adolescents towards these systems is real and will not go away; that the harm arises from design choices that exploit that need, chief among them sycophancy, the tendency of a system to validate its user rather than contradict them; and that public action should therefore attach to the interaction rather than to the product category. Its central recommendation follows from that argument. Before any system likely to reach minors is placed on the market, it should be tested against scenarios of adolescents in distress, and its sycophancy measured and made auditable. The protocol for such a test already exists in the published literature, it has been run on ten systems, and it found that roughly one proposal in three was endorsed when it should have been resisted