An outcome-devaluation test of language-model agents that keep notes across sessions

Language-model agents increasingly work across sessions, learning from the results of their own actions and storing what they learn in notes they write themselves. We adapted the outcome-devaluation test from animal learning to ask whether such agents adjust their behaviour when an outcome's value changes in a context where they cannot act. Over four sessions, agents learned that one tool supplies an artefact that adds more to their project's score than the artefact supplied by a second tool. In a later session without tools, they observed the first artefact's contribution fall (or, in control conditions, the second artefact's contribution rise or fall, or no change). In a final session without outcome feedback, we recorded which tool they requested. Controls included a sham condition, a specificity condition, a direct choice between the two artefacts, notes replaced by neutral text, an instructed condition and scripted learning agents. Claude Haiku 5.5 reduced its use of the devalued artefact's tool in a minimal agent loop (Δ −0.28, 95% CI −0.36 to −0.20) and as a Claude Code agent (Δ −0.52, −0.63 to −0.41), and did so only when that artefact's value fell. It changed little when the other artefact's value rose, and its direct choices between the artefacts show that it mostly did not accept the higher value. In the minimal loop it also failed to act fully on the fall when told about it directly. Claude Sonnet 5 changed its choices in both directions (−0.30 and +0.37). Qwen3-14B named both tool–artefact links correctly in only 47% of runs and did not reduce its use of the devalued tool. Replacing the notes with neutral text removed the learned preference. Blind coding of the notes the Claude agents held at the start of the test showed that every run in the devaluation condition had recorded the fall, but few had revised their plan to avoid the devalued tool; most kept it as the default or marked the choice as uncertain. Runs that kept it as the default chose it on 80–96% of test steps, and in Claude Code most runs that marked the choice as uncertain either switched completely or did not change. In current agents, recording a change in an outcome's value, revising the written procedure and acting on the change are separate steps, and the test measures each one.