Where should a rule live? Prose, skills, hooks and graphs in a coding agent, with the price of each
Practitioners steer coding agents with prose rules, numbered checklists, on-demand skills, lifecycle hooks and state machines, and each of these has mostly been measured alone. We compare them side by side on the same procedures in Claude Code, in a sequence of pre-registered experiments (about 5,700 counted runs, scored on the state of the files, with token cost by class). On a short, complete procedure Claude Sonnet 5.5 followed plain prose in 40 of 40 runs. Thirteen other mechanisms reached 39 or 40 of 40 at 0.6 to 2.4 times the cost of prose, and a fifteenth, a script-run graph, reached 40 of 40 once a defect in its driver was repaired; a sensitivity control showed that the bench detects failure. The mechanism mattered in three places. A stop gate recovered the items it checked, on a deliberately incomplete instruction and on an exact output form that the smaller Claude Haiku 5.5 read differently, and left the other items as they were. For a standing rule that must precede an action, the place of the rule did not matter and the moment of delivery did: an always-loaded file, a skill whose description names the action, a hook injecting the rule on any call that touches the folder and a hook refusing the first write each held the rule in 40 of 40 runs, against 2 to 7 of 40 for three deliveries that arrive with or after the action. For Haiku 5.5, the wording and heading of a constraint in a resume file moved compliance from 10% to 90%. A second procedure with a branch gave the same picture on both models. Three open-weight models run locally did not show the wording effect, and the smallest never loaded a skill. The experiments were designed, run, scored and first written up by AI agent sessions (Claude Code, models of the Claude family) working under the direction of the author; see the statement at the end of the paper. The replication package (code, cases, pre-registrations and per-run tables) is in the attached zip file.
Authors
- Yannis Coelo
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-15
- DOI
- https://doi.org/10.5281/zenodo.24248574
- Primary Topic
- Software Engineering Research
- Type
- article
- Field-Weighted Citation Impact
- 0.00