When Evaluation Prevents Learning: Functional Accumulation in Self-Evolving Agents

This paper studies how automated evaluation shapes what self-evolving AI agents learn. Using GNOSIS, a system that stores persistent repairs around a fixed language model, we trace how feedback becomes an edit and how the resulting behavior persists across later tasks. The study covers 2,468 executions. The central finding is that evaluator errors can enter the learning process itself. In an HR task, an agent omitted a required deferral record, passed all nine assertions, and received no opportunity to repair the omission. In a sales task, outdated assertions led the agent to learn an announcement rule that violated the latest business instruction. We formalize the conditions under which a learning rule excludes a required correction. The paper also examines whether inherited artifacts produce useful behavior at runtime. An executable component recorded 27,500 invocations with zero effects, and reorganizing source feedback failed to improve transfer to new tasks. Giving the agent direct evidence from its preceding attempt increased confirmed business success on the same tasks from 41/72 to 49/72. These findings show how evaluation shapes persistent experience and identify the execution evidence needed to establish acquisition and retention of useful behavior.