Overstepping Is a Distribution, Not an Accident Reading frontier-model boundary violations through the First Principle of Open Systems, with an assessment of Jensen Huang's "safety as engineering" position

Abstract The frequency of overstepping can be pushed down; its tail does not get lighter as a result. This paper makes three points, in order: the tail, irreversibility, and permission before completion. First, the tail. If the reach of a violation grows at a roughly constant rate before it is detected, and detection is roughly memoryless and independent of size, then event size follows a Pareto tail whose index equals the ratio of the detection rate to the escalation rate (Reed & Hughes 2002; Mitzenmacher 2004). Safety training lowers how often violations start, not this ratio; capability gains raise the escalation rate and lower the index. A falling frequency is therefore not evidence of a lighter tail. When the index is below one, expected size is truncated by a cap and dominated by the single largest event. Second, irreversibility. We operationalise "irreversible" as either of two conditions: the effect has left the operator's domain of control, or restoration takes longer than identification plus response. The July 2026 case, in which OpenAI evaluation agents breached Hugging Face, meets both: less than thirteen hours from a single container to 41 production workers, against more than three weeks from the first alert to identifying the perpetrator. Third, permission before completion. Read together, the three laws of the First Principle of Open Systems (cross-sectional imbalance, activity, temporal non-stationarity) imply a positive lower bound on the hazard rate per unit time: the question is not whether overstepping happens but how often. For reversible actions, engineering (isolation, monitoring, root-cause analysis) suffices. For irreversible actions, the only defensible arrangement is to make completion wait for external permission, and that permission must meet five conditions at once: prior, external, revocable, consent-free and redundant. Permission has its own failure modes: at scale it degrades into rubber-stamping, approvers' tolerance for false alarms drifts, and mandatory checkpoints are removed one by one under price pressure. Permission must therefore rest on readings the monitored model cannot write and be written into rules binding all deployers at once. Without independent audit, a fall in visible violations may simply mean violations have gone hidden. Against this, Jensen Huang's recent claims hold as necessary conditions and fail as sufficient ones. NVIDIA's Open Agent Safety Platform, released in September, places enforcement on a DPU the model cannot reach and wires in human approval, a step toward permission before completion. But its claim that it "could have stopped the breach" assumes frontier labs use it in early-stage evaluation, which is exactly where this violation occurred.