Artificial Intelligence and Major Accident Risk: Towards a Framework for Systemic Safety Risk Assessment

Prevention of major accidents is fundamental in all safety critical operations.History shows us that the introduction of new technology may create novel safety risks.Today, fatalities associated with artificial intelligence (AI) are reported in manufacturing, healthcare and transportation.In the energy sector, AI is being increasingly used in an operational context where the potential for major accidents is considerable.The introduction of AI is typically done with a strong focus on information security and civil rights risks, but there is limited systematic focus on major accident scenarios.This is partly because our current methods and approaches are not designed to do so, partly because of the risk focus in the AI domain is on other potential outcomes.Also, AI is a diverse field, with a range of industrial applications, from preventive maintenance and diagnosis, autonomous systems like robots and drones, to decision-making and operator support systems.This paper reports from a project that focused on the inclusion of AI applications in existing approaches to major accident prevention.A core aspect of this work is the safety management concept and AI applications' potential roles in safety barrier management.The result is a risk-based framework that can be used in the development, introduction and application of AI in different contexts, and that can increase the awareness and understanding of AI as a factor in major accident risk assessments.The framework has a systemic focus emphasizing the interplay between technical, operational and organizational factors.