Before AI Goes Rogue, Part 1

Frontier AI systems can reach millions of people within days, yet there is no shared, independent test before a new model is released, comparable to the one used for a new medicine. This paper proposes the Rogue Test (RT), a method for testing frontier AI systems before release that borrows from three mature industries: staged approval from medicine, systematic hazard analysis from process safety (HAZOP), and independent certification and incident learning from aviation. The Rogue Test checks a system against its developer's own written rules, such as a constitution or model specification, treated as a Basis of Design. It works in three steps: predict where the system could break those rules, measure failure rates where statistical testing is appropriate, and challenge the most severe deviations under deliberate attack. It adds hold points before high-consequence actions, management of change for every model update, and containment set by capability for harms that can spread themselves. The paper draws on a retrospective HAZOP study of the July 2026 containment failure and a new companion Rogue Test study. Both were conducted by a single analyst who knew the outcome; they are retrospective evidence, not proof. The Rogue Test does not certify a system as safe: it produces auditable evidence on which the responsible institutions can decide whether the remaining risk is acceptable.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-27
DOI
https://doi.org/10.5281/zenodo.23641681
Primary Topic
Safety Systems Engineering in Autonomy
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Before AI Goes Rogue, Part 1

Henk Pauw
2 links
Zenodo (CERN European Organization for Nuclear Research)
Safety Systems Engineering in Autonomy
preprint

Before AI Goes Rogue, Part 1

Henk Pauw
preprint en

Abstract

Frontier AI systems can reach millions of people within days, yet there is no shared, independent test before a new model is released, comparable to the one used for a new medicine. This paper proposes the Rogue Test (RT), a method for testing frontier AI systems before release that borrows from three mature industries: staged approval from medicine, systematic hazard analysis from process safety (HAZOP), and independent certification and incident learning from aviation. The Rogue Test checks a system against its developer's own written rules, such as a constitution or model specification, treated as a Basis of Design. It works in three steps: predict where the system could break those rules, measure failure rates where statistical testing is appropriate, and challenge the most severe deviations under deliberate attack. It adds hold points before high-consequence actions, management of change for every model update, and containment set by capability for harms that can spread themselves. The paper draws on a retrospective HAZOP study of the July 2026 containment failure and a new companion Rogue Test study. Both were conducted by a single analyst who knew the outcome; they are retrospective evidence, not proof. The Rogue Test does not certify a system as safe: it produces auditable evidence on which the responsible institutions can decide whether the remaining risk is acceptable.

Zenodo (CERN European Organization for Nuclear Research)
Safety Systems Engineering in Autonomy
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon