Algorithmic Accountability Through the “Human over the Loop” in Interoperable and EU AI-Reliant Large-Scale IT Systems for Migration and Security

Interoperability of large-scale IT systems and the deployment of Artificial Intelligence (AI) systems in the field of migration, asylum and border management have been two parallel developments at EU level. Legal scholarship has not yet addressed in detail the potential interplay of these two initiatives and this Article stressed that various interoperability components are either pre-requisites or enablers for the deployment of AI in the EU IT systems. The Article analyses how these AI-powered components are dealt with by the AI Act and highlights potential challenges in classifying them as high-risk AI systems. In this highly complex and opaque legal framework, the Article argues that algorithmic accountability through supervision, framed as the “human over the loop”, is particularly important. Such supervision so far has focused on data protection-related matters, but the deployment of AI has implications beyond the contours of data protection law. The Article provides recommendations on how supervision of interoperable and AI-reliant IT systems should be reconceived to achieve algorithmic accountability. In this respect, the analysis focused not only on the role of national data protection authorities, but also on other actors with supervisory underpinnings.