High-Risk Artificial Intelligence

Given the immense opportunities and significant risks associated with AI, the topic has increasingly drawn attention from academia, industry, and legislators.As such, in April 2024, following three years of trilateral negotiations between the European Commission, the European Council, and the European Parliament, the European Union introduced a landmark regulation on AI.This so-called ''AI Act'' aims to establish the first comprehensive legal framework governing the use of AI technologies in the European Union (European Commission 2021; European Parliament 2024).A central component of the AI Act is its definition of four risk classes for AI systems (see Fig. 1).With this risk classification, the AI Act puts particular emphasis on the regulation of high-risk AI systems (i.e., those AI systems that could impact and endanger the health, safety, or fundamental rights of individuals).These systems are subject to rigorous oversight, including mandatory internal conformity assessments by the providers and, in special cases, external reviews by notified bodies (European Parliament 2024;Hupont et al. 2023).Despite such regulatory measures being put forward, numerous questions remain open, especially regarding the integration of high-risk AI in IS.While high-risk AI in IS hold tremendous promise for advancements in business and society, they also bring forth complex issues, highlighting technical and societal dilemmas and the need for balanced trade-offs to resolve such dilemmas (Thiebes et al. 2021).As a socio-technical discipline that integrates insights from computer science, management, and other domains to drive technological innovation in business and society, IS scholarship and practice play a critical role in addressing the challenges surrounding the responsible and sustainable development