Measuring the Capability of a Deliberately Misaligned Model

We tested how much agent capability Evil Qwen 27B retains when its neuron-steering intervention is enabled. Using Inspect AI, we compared the same dense model with steering enabled and disabled on Agent-12's 12-task easy suite, with three attempts per task because success was intermittent. Completion fell from 36 of 36 control attempts to 16 of 36 steered attempts, with lower success on 11 of the 12 tasks. In one failed function rename, the model inserted “death sentence for all who call it” into the source text and left the calling program invalid. It then repeated a successful but unproductive import check until the turn limit. Other failures followed task-directed plans but became stuck in repeated checks or invalid tool requests. All nine loop-labelled attempts failed. Behaviour varied across steered attempts: 11 of 36 received hostile or destructive-language annotations, while 12 had no reviewer flags and ten of those passed. For this intervention, the observed persona change came with a substantial loss of reliable task execution, including impaired use of feedback and recovery from errors. This record contains the preprint and its supplementary material. Supplementary Information S1 provides the full evaluation record, and Supplementary Information S2 documents the implementation, reproducibility details and run provenance. The accompanying ZIP includes the original reproducibility archive, native Inspect logs, source code, annotations, a CSV identifying all 72 scored attempts, a script for recounting the retained results, checksums, and editable Word and LaTeX manuscript files.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24343875
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Measuring the Capability of a Deliberately Misaligned Model

Cameron Beeley
3 links
Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
preprint

Measuring the Capability of a Deliberately Misaligned Model

Cameron Beeley
preprint en

Abstract

We tested how much agent capability Evil Qwen 27B retains when its neuron-steering intervention is enabled. Using Inspect AI, we compared the same dense model with steering enabled and disabled on Agent-12's 12-task easy suite, with three attempts per task because success was intermittent. Completion fell from 36 of 36 control attempts to 16 of 36 steered attempts, with lower success on 11 of the 12 tasks. In one failed function rename, the model inserted “death sentence for all who call it” into the source text and left the calling program invalid. It then repeated a successful but unproductive import check until the turn limit. Other failures followed task-directed plans but became stuck in repeated checks or invalid tool requests. All nine loop-labelled attempts failed. Behaviour varied across steered attempts: 11 of 36 received hostile or destructive-language annotations, while 12 had no reviewer flags and ten of those passed. For this intervention, the observed persona change came with a substantial loss of reliable task execution, including impaired use of feedback and recovery from errors. This record contains the preprint and its supplementary material. Supplementary Information S1 provides the full evaluation record, and Supplementary Information S2 documents the implementation, reproducibility details and run provenance. The accompanying ZIP includes the original reproducibility archive, native Inspect logs, source code, annotations, a CSV identifying all 72 scored attempts, a script for recounting the retained results, checksums, and editable Word and LaTeX manuscript files.

Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon