Thresholds That Travel: Adaptive Alarm Thresholds Let Attacks Through in ICS Anomaly Detection

Anomaly detectors for industrial control systems (ICS) raise an alarm when a score crosses a threshold. The threshold is calibrated on clean data and must then hold a false-alarm budget for months, so deployments often let it adapt: they recalibrate it daily, track a running quantile, or refit an extreme-value tail as data arrive. Each rule learns from part of its input, and we ask what happens when attacks land there. In two pre-registered studies on data that no earlier analysis in our pipeline had opened, a symmetric quantile tracker lost recall against a frozen threshold. It detected 20 of HAI 20.07's 38 attacks against 36 (difference -0.42, 95% interval [-0.58, -0.26]) and 6 of BATADAL's 14 against 14 (-0.57, [-0.79, -0.29]), and every attack it caught, the frozen threshold caught too. Rerun post-hoc with an update that saw only the attack samples, it lost as much on HAI 20.07 (20 of 38) and nearly as much on BATADAL (7 of 14). A short analysis explains why the loss lasts: a tracker calibrated to HAI 20.07's false-alarm rate needs 23 exceedance-free hours to undo one minute above its threshold, whatever its step size. The BATADAL loss survives a stricter count in which an attack is detected only if an alarm begins inside it; the HAI 20.07 loss does not (-0.16, [-0.37, 0.05]), because the frozen threshold there was in alarm for 39% of the unlabelled time and was credited with attacks that arrived inside alarms already running. Daily recalibration lost recall on HAI 20.07 under both counts. On three earlier HAI releases (exploratory) we add two attacks and stronger baselines. Twelve two-minute nuisance alarms a day left daily recalibration and both trackers catching 1 to 4 of 30 later, just-detectable setpoint steps that the frozen threshold caught every time, and a closed-loop attacker who held the score just under the current threshold drove streaming extreme-value thresholds (SPOT, DSPOT) upward without raising an alarm. The registered test of that attack on BATADAL could not be evaluated: fitted to a heavy-tailed calibration score, method-of-moments SPOT and DSPOT started 49 and 46 times above calibration before any attack, and DSPOT already missed the attack in every control, so that study's primary rule failed. Adaptive rules that leave alarm episodes out of their update or cap their daily rise came within one event of the frozen threshold's recall on those releases, and a standard maximum-likelihood SPOT was blinded by the closed-loop attacker in every placement of its 24-hour campaign. A threshold that only tightens itself cannot miss an event the frozen threshold catches; on BATADAL it cost 2.43 times the false-alarm budget, against 0.70 for the frozen threshold. We argue that an alarm threshold may tighten itself but should loosen only with operator approval, and we release the registrations, ledger and code. Preprint. The paper sources, registrations, data-access ledger, harness, verifiers and result files are in https://github.com/VyomKulshrestha/Ferrum-OS under docs/research.