AI safety and regulatory capture

Abstract Researchers, politicians, and the general public support safety regulations on the production and use of AI technology. Yet regulations on new technology are susceptible to the harmful phenomenon of regulatory capture, in which organizations and institutions with economic or political power exert that power to use regulations to unjustly enrich themselves. Only a few authors have tried to raise the alarm about regulatory capture in AI safety and even fewer have described the problem and its implications in detail. Therefore, this paper has three related goals. The first goal is to argue for caution: AI safety is a field with enormous potential for such regulatory capture. Second, this paper explores, in detail, a variety of harms and injustices that captured AI-safety regulations are likely to create. The third goal, in the penultimate section, is to review and critique a few proposals that might mitigate the problem of regulatory capture of AI safety.