Mapping LLM-agent reliability under chaos: a cross-architecture study of structured recovery playbooks

Large language model (LLM) agents increasingly execute multi-step business workflows, yet their reliability under operational chaos (transient tool failures such as timeouts, rate limits and server errors) is poorly characterised. We use chaos engineering as a scientific instrument: seeded fault injection at the agent-tool boundary. We study eleven open and frontier LLMs (0.8B to 355B parameters; dense, per-layer-embedding, hybrid Mamba2-Transformer and mixture-of-experts architectures), three emulated agent-tool interface styles (HTTP, MCP, A2A) and ten failure rates from 0% to 30%, comparing in 128,000 paired runs an unguided agent with the same model given a 69-word recovery playbook. Over the nine functioning models, the median gain in success rate at 30% chaos is +43.5 percentage points (bootstrap 95% CI [40.0, 55.0]); in exact paired McNemar tests with Holm correction the effect is significant in every HTTP cell at failure rates of 5% and above, and no model is harmed under the transient faults we inject. Much of the effect is explained by retry behaviour: several models rarely retry a failed call on their own, and the playbook induces them to. The gain depends on capability and chaos rate: at 30% chaos the largest gains are in the mid-range and the frontier model gains less, while at 10% the gain is not monotonic in size. The playbook adds 13-69% more tokens per run (mean 42%) and 0.57 LLM calls on average. We document a capacity floor and a structural-rescue artefact, and release the rig, playbooks, dataset and scripts. The measurement also relates to the EU AI Act requirement that high-risk systems be resilient to errors and faults arising from interaction with other systems.