Measuring the Capability of a Deliberately Misaligned Model
We tested how much agent capability Evil Qwen 27B retains when its neuron-steering intervention is enabled. Using Inspect AI, we compared the same dense model with steering enabled and disabled on Agent-12's 12-task easy suite, with three attempts per task because success was intermittent. Completion fell from 36 of 36 control attempts to 16 of 36 steered attempts, with lower success on 11 of the 12 tasks. In one failed function rename, the model inserted “death sentence for all who call it” into the source text and left the calling program invalid. It then repeated a successful but unproductive import check until the turn limit. Other failures followed task-directed plans but became stuck in repeated checks or invalid tool requests. All nine loop-labelled attempts failed. Behaviour varied across steered attempts: 11 of 36 received hostile or destructive-language annotations, while 12 had no reviewer flags and ten of those passed. For this intervention, the observed persona change came with a substantial loss of reliable task execution, including impaired use of feedback and recovery from errors. This record contains the preprint and its supplementary material. Supplementary Information S1 provides the full evaluation record, and Supplementary Information S2 documents the implementation, reproducibility details and run provenance. The accompanying ZIP includes the original reproducibility archive, native Inspect logs, source code, annotations, a CSV identifying all 72 scored attempts, a script for recounting the retained results, checksums, and editable Word and LaTeX manuscript files.
Authors
- Cameron Beeley
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24343875
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint