Assessing Compliance in Child-Facing High-Risk AI IoT Devices: Legal Obligations Under the EU’s AI Act and GDPR

The rapid and ongoing adoption of smart home products, coupled with the increasing integration of artificial intelligence (AI), particularly in these products, is an undeniable reality. However, as both technologies converge, they also give rise to a range of significant concerns. The EU’s recent AI Act specifically addresses the challenges associated with the use of AI technology. In this study, we examine three AI-integrated products with toy capabilities that are sold in Spain, serving as a case study for the EU market of smart home devices that incorporate AI. Our research aims to identify potential compliance issues with both the AI Act and the General Data Protection Regulation (GDPR). Our results reveal a clear and worrying gap between the existing legislation and the functionalities of these devices. Using a normal user’s approach, we find that the privacy policies for these products, whose features make them high-risk AI systems, AI systems with systemic risk, or both as per the AI Act, fail to provide any information about AI usage, particularly of ChatGPT, which they all integrate. This raises significant concerns, especially as the market for such products will continue to grow. Without rigorous enforcement of existing legislation, the risk of misuse of sensitive personal information becomes even greater, making strict regulatory oversight essential to ensure user protection.