From Recursive Self-Improvement to Recursive Safety Improvement: The Urgent Pivot Towards Human and Environmental Protection in AI Development

This paper argues that recursive capability improvement in artificial intelligence, the use of capable systems to build more capable successors, creates a binding obligation to improve safety at the same recursive pace, and that the safety in question must extend beyond the model to the people and environments that bear its consequences. It identifies the governance gap as a mismatch of loops: developers now run a capability loop and have begun a technical safety loop that hardens models against misuse and failure, but no comparable loop continuously identifies, measures, and reduces the harms that advancing systems impose on workers, households, communities, infrastructure, and ecosystems. Drawing on an independent evaluator's review of a frontier laboratory's risk report, on published frameworks for recursive safety improvement, and on synthetic-population evaluation infrastructure, it finds that the leading capability loop has reached its own early-warning threshold, that the evaluations meant to track it have saturated, and that technical safety gains do not establish protection for affected people. It defines corresponding safety advancement as demonstrated protection proportionate to new capabilities and their foreseeable consequences, specifies six questions every development cycle must answer, and states a stop condition: developers must not advance a system beyond the level at which they can demonstrate adequate protection against the material risks its capabilities introduce. It holds that affected people must help define harm and retain authority to challenge or refuse interventions, and that the governing principle survives disagreement about any particular enforcement instrument. No recursive capability advancement without recursive safety advancement.