Where should a rule live? Prose, skills, hooks and graphs in a coding agent, with the price of each

Practitioners steer coding agents with prose rules, numbered checklists, on-demand skills, lifecycle hooks and state machines, and each of these has mostly been measured alone. We compare them side by side on the same procedures in Claude Code, in a sequence of pre-registered experiments (about 5,700 counted runs, scored on the state of the files, with token cost by class). On a short, complete procedure Claude Sonnet 5.5 followed plain prose in 40 of 40 runs. Thirteen other mechanisms reached 39 or 40 of 40 at 0.6 to 2.4 times the cost of prose, and a fifteenth, a script-run graph, reached 40 of 40 once a defect in its driver was repaired; a sensitivity control showed that the bench detects failure. The mechanism mattered in three places. A stop gate recovered the items it checked, on a deliberately incomplete instruction and on an exact output form that the smaller Claude Haiku 5.5 read differently, and left the other items as they were. For a standing rule that must precede an action, the place of the rule did not matter and the moment of delivery did: an always-loaded file, a skill whose description names the action, a hook injecting the rule on any call that touches the folder and a hook refusing the first write each held the rule in 40 of 40 runs, against 2 to 7 of 40 for three deliveries that arrive with or after the action. For Haiku 5.5, the wording and heading of a constraint in a resume file moved compliance from 10% to 90%. A second procedure with a branch gave the same picture on both models. Three open-weight models run locally did not show the wording effect, and the smallest never loaded a skill. The experiments were designed, run, scored and first written up by AI agent sessions (Claude Code, models of the Claude family) working under the direction of the author; see the statement at the end of the paper. The replication package (code, cases, pre-registrations and per-run tables) is in the attached zip file.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24248574
Primary Topic
Software Engineering Research
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Where should a rule live? Prose, skills, hooks and graphs in a coding agent, with the price of each

Yannis Coelo
Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
article

Where should a rule live? Prose, skills, hooks and graphs in a coding agent, with the price of each

Yannis Coelo
article en

Abstract

Practitioners steer coding agents with prose rules, numbered checklists, on-demand skills, lifecycle hooks and state machines, and each of these has mostly been measured alone. We compare them side by side on the same procedures in Claude Code, in a sequence of pre-registered experiments (about 5,700 counted runs, scored on the state of the files, with token cost by class). On a short, complete procedure Claude Sonnet 5.5 followed plain prose in 40 of 40 runs. Thirteen other mechanisms reached 39 or 40 of 40 at 0.6 to 2.4 times the cost of prose, and a fifteenth, a script-run graph, reached 40 of 40 once a defect in its driver was repaired; a sensitivity control showed that the bench detects failure. The mechanism mattered in three places. A stop gate recovered the items it checked, on a deliberately incomplete instruction and on an exact output form that the smaller Claude Haiku 5.5 read differently, and left the other items as they were. For a standing rule that must precede an action, the place of the rule did not matter and the moment of delivery did: an always-loaded file, a skill whose description names the action, a hook injecting the rule on any call that touches the folder and a hook refusing the first write each held the rule in 40 of 40 runs, against 2 to 7 of 40 for three deliveries that arrive with or after the action. For Haiku 5.5, the wording and heading of a constraint in a resume file moved compliance from 10% to 90%. A second procedure with a branch gave the same picture on both models. Three open-weight models run locally did not show the wording effect, and the smallest never loaded a skill. The experiments were designed, run, scored and first written up by AI agent sessions (Claude Code, models of the Claude family) working under the direction of the author; see the statement at the end of the paper. The replication package (code, cases, pre-registrations and per-run tables) is in the attached zip file.

Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Where should a rule live? Prose, skills, hooks and graphs in a coding agent, with the price of each — Yannis Coelo · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS