More Governable or Less Corrigible? What Giving AI Agents a Stake Does to Oversight
AI agents are starting to accumulate things they could lose: persistent memory, money in wallets, reputations, and continued deployment. Two lines of work predict opposite effects. Work on corrigibility predicts that an agent with more to lose will resist correction and shutdown more. Work on legal rights for AIs predicts that an agent with more to lose has more reason to cooperate and stay within our institutions. A third argues that a designed stake changes nothing, because such agents can be copied and reset. Or the first two could both hold at once, as a trade-off. This research proposal sets out how to test which. Study 1 establishes a baseline with today’s agents, which have nothing of their own at stake: does it matter where a sanction’s cost lands? Study 2 gives agents a stake by design and measures two things together: whether sanctions work better on them, and whether they still accept correction. The proposal reports no data. It shows what each plausible result would mean for how agents should be designed and governed, and closes by sketching how a third study could detect the case that matters most: an agent guarding a stake that no one gave it.
Authors
- Nitin Kashyap
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-14
- DOI
- https://doi.org/10.5281/zenodo.24215757
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- preprint