When, Where, and How Experience Becomes Structure: Mechanisms for Recursive Self-Improvement in LLM Agents

Recursive self-improvement (RSI) studies how AI systems modify themselves and carry those changes into the generation of subsequent improvements. LLM agents can pursue this process by using interaction experience to revise Prompt, Memory, Skill, and Harness in some cases while keeping the task-execution model’s weights fixed. However, execution feedback does not directly specify which component should change, and a locally successful revision may not remain useful across tasks or subsequent improvement cycles. Understanding how experience becomes persistent agent structure therefore requires examining when evidence justifies a change, where that change should act, and how it is constructed, verified, and retained. This survey connects four components--Prompt, Memory, Skill, and Harness--with seven analytical functions: Collect, Evaluate, Diagnose, Abstract, Update, Verify, and Consolidate. The resulting framework supports comparisons of instruction revision, memory management, skill construction and maintenance, and Harness evolution through their update loops. These comparisons examine how mechanisms use observations to construct candidate changes, distinguishing feedback that triggers revision from evidence that supports retention. The analysis further addresses coordinated updates across components and the revision or retirement of retained components as tasks, tools, and models change. Finally, we identify cross-component attribution, long-term effectiveness, and independent validation as key challenges, and outline controlled and longitudinal evaluations of task-performance gains, the contribution of inherited state, and the effectiveness of subsequent updates.