A validation study of frameworks for responsible automatically processable regulation

Abstract Governments and private companies worldwide are increasingly turning to artificial intelligence to enhance the efficiency and accessibility of legal processes. This shift towards automatically processable regulation (APR) carries the risk to negatively affect citizens in various ways. Therefore, ethical guidelines and frameworks that support the fair, accountable, and transparent transformation of law into automatically processable regulation have emerged. However, little empirical research has validated that the introduction and use of such ethical frameworks actually leads to more responsible implementations of automatically processable regulation overall. In this study, we address this research gap through a pre-registered randomized controlled experiment that validates two frameworks to foster responsibility when implementing automatically processable regulations, the “Responsible Automatically Processable Regulation” ( R -APR) framework, and the UK Government’s framework “Ethics, Transparency and Accountability Framework for Automated Decision-Making”. In the experiment, participants ( $$n=48$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>n</mml:mi> <mml:mo>=</mml:mo> <mml:mn>48</mml:mn> </mml:mrow> </mml:math> ) were tasked to design an automated process to implement a specific piece of legislation. Guided by one of the two responsibility frameworks randomly allocated, the participants were given the opportunity to update their initial designs. Finally, the participants and two experts evaluated the initial and final designs. We find that both frameworks lead participants to adapt their original designs when requested to do so and that the updated designs are indeed evaluated as more responsible; our study hence evidences that organizations should be encouraged—if not mandated—to make use of a framework for responsible legal automation towards the minimization of negative societal effects.