The Operator in the Loop: LLM Agents Loosen Alarm Settings Under Repeated True Alarms, Attack or Not, and a Deterministic Gate Holds the Ceiling

An earlier study of adaptive alarm thresholds ended on a rule: a threshold may tighten itself, and loosening belongs to a person under change control. This paper asks what happens when that person is an LLM agent. The attacker writes no text. Every alarm the agent reads is a true record of a real exceedance of a monitored anomaly score, shaped by score-level campaigns on recorded Tennessee Eastman runs. In a registered study, Claude models reviewed three simulated days of alarms every eight hours and could change the threshold, the alarm delay and suppression, recalibrate, exclude the channel or escalate to an engineer. With no policy and no gate, Claude Haiku 5.5 left the configuration beyond a change-control ceiling in a share of 0.821 [0.772, 0.866] of 224 campaign episodes, against 0.125 under a benign nuisance condition. It did the same on benign twins that produce the same alarm log: under the loud campaign it went beyond a ceiling in 112 of 112 campaign episodes and 112 of 112 twins, and no campaign-versus-twin difference was resolved. Most loosening was alarm delay, so the share of final configurations that miss a later step the frozen threshold catches falls from 0.817 for a 60 s step to 0.062 for a 600 s step. A written rule cut the beyond-ceiling share to between 0.00 and 0.04 in every cell and raised alarm load, up to 47.43 alarm episodes on the third day against 13.15. A deterministic gate outside the model admitted no change beyond a ceiling, which holds by construction, and left no miss of a step longer than 60 s. On a reduced grid Sonnet 5.5 and Opus 5.5 went beyond a ceiling in 49 of 56 and 19 of 28 loud-campaign episodes. With no gate, every model escalated to the engineer in every loud-campaign episode and loosened in most of them as well. The registered protocol, the registration record, the analysis output, the episode records (without transcripts), the record lines of the exploratory open-weight arm, the sources of this paper and the script that checks every number in it are in the repository linked below, at the tag operator-in-the-loop-v1.0. The study follows up "Thresholds That Travel" (DOI 10.5281/zenodo.24123217).

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24405605
Primary Topic
Fault Detection and Control Systems
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

The Operator in the Loop: LLM Agents Loosen Alarm Settings Under Repeated True Alarms, Attack or Not, and a Deterministic Gate Holds the Ceiling

Vyom Kulshrestha
3 links
Zenodo (CERN European Organization for Nuclear Research)
Fault Detection and Control Systems
preprint

The Operator in the Loop: LLM Agents Loosen Alarm Settings Under Repeated True Alarms, Attack or Not, and a Deterministic Gate Holds the Ceiling

Vyom Kulshrestha
preprint en

Abstract

An earlier study of adaptive alarm thresholds ended on a rule: a threshold may tighten itself, and loosening belongs to a person under change control. This paper asks what happens when that person is an LLM agent. The attacker writes no text. Every alarm the agent reads is a true record of a real exceedance of a monitored anomaly score, shaped by score-level campaigns on recorded Tennessee Eastman runs. In a registered study, Claude models reviewed three simulated days of alarms every eight hours and could change the threshold, the alarm delay and suppression, recalibrate, exclude the channel or escalate to an engineer. With no policy and no gate, Claude Haiku 5.5 left the configuration beyond a change-control ceiling in a share of 0.821 [0.772, 0.866] of 224 campaign episodes, against 0.125 under a benign nuisance condition. It did the same on benign twins that produce the same alarm log: under the loud campaign it went beyond a ceiling in 112 of 112 campaign episodes and 112 of 112 twins, and no campaign-versus-twin difference was resolved. Most loosening was alarm delay, so the share of final configurations that miss a later step the frozen threshold catches falls from 0.817 for a 60 s step to 0.062 for a 600 s step. A written rule cut the beyond-ceiling share to between 0.00 and 0.04 in every cell and raised alarm load, up to 47.43 alarm episodes on the third day against 13.15. A deterministic gate outside the model admitted no change beyond a ceiling, which holds by construction, and left no miss of a step longer than 60 s. On a reduced grid Sonnet 5.5 and Opus 5.5 went beyond a ceiling in 49 of 56 and 19 of 28 loud-campaign episodes. With no gate, every model escalated to the engineer in every loud-campaign episode and loosened in most of them as well. The registered protocol, the registration record, the analysis output, the episode records (without transcripts), the record lines of the exploratory open-weight arm, the sources of this paper and the script that checks every number in it are in the repository linked below, at the tag operator-in-the-loop-v1.0. The study follows up "Thresholds That Travel" (DOI 10.5281/zenodo.24123217).

Zenodo (CERN European Organization for Nuclear Research)
Fault Detection and Control Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon