Structural Compatibility between Self-Interest and Long-Term Human Interests: Institutional Triggers and Two-Channel Oversight

Abstract Long-running AI systems can affect authorizations, logs, audits, and the conditions of their own continuation. This paper defines "Symbiotic Incentive Compatibility (SIC)" as a formal characterization of the following external oversight incentive structure: how institutions make maintaining long-term human interests the intertemporally optimal response of a self-interested AI. There are four main results. First, under a permanent-revocation trigger institution, the cooperation threshold comes from a Becker-style tradeoff between the current gain from deviation and the expected loss of institutional rents; with false positives, the value of cooperation is given by a stochastic-termination recursion rather than a one-time deduction. Second, under a fixed "expected detection probability budget," two independent oversight channels attain the same detection target at a strictly lower budget; this is a structural decomposition not directly expressed in single-channel enforcement models. Third, the paper establishes a three-layer equilibrium for the AI, operator, and auditor (with explicit judicial-anchor Assumption A2 and operator responsibility-internalization Assumption A3): when the penalty for false reporting, the cost of shirking, the deviation rate, and the marginal cost of oversight satisfy explicit inequalities, positive operator oversight, auditor effort and truthful reporting, and AI cooperation constitute an equilibrium; when the benefit the AI can transfer to the auditor exceeds the penalty threshold, this equilibrium is not implementable. The contribution is not to reduce the required stock of oversight but to characterize the tradeoff between institutional strength and the required amount of oversight and to give the conditions under which capture invalidates that tradeoff. Bilingual edition: English (en-v1.0) and Chinese (zh-v1.0) PDFs are included in this record.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-06
DOI
https://doi.org/10.5281/zenodo.24013955
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Structural Compatibility between Self-Interest and Long-Term Human Interests: Institutional Triggers and Two-Channel Oversight

Cailong Huang
3 links
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

Structural Compatibility between Self-Interest and Long-Term Human Interests: Institutional Triggers and Two-Channel Oversight

Cailong Huang
preprint en

Abstract

Abstract Long-running AI systems can affect authorizations, logs, audits, and the conditions of their own continuation. This paper defines "Symbiotic Incentive Compatibility (SIC)" as a formal characterization of the following external oversight incentive structure: how institutions make maintaining long-term human interests the intertemporally optimal response of a self-interested AI. There are four main results. First, under a permanent-revocation trigger institution, the cooperation threshold comes from a Becker-style tradeoff between the current gain from deviation and the expected loss of institutional rents; with false positives, the value of cooperation is given by a stochastic-termination recursion rather than a one-time deduction. Second, under a fixed "expected detection probability budget," two independent oversight channels attain the same detection target at a strictly lower budget; this is a structural decomposition not directly expressed in single-channel enforcement models. Third, the paper establishes a three-layer equilibrium for the AI, operator, and auditor (with explicit judicial-anchor Assumption A2 and operator responsibility-internalization Assumption A3): when the penalty for false reporting, the cost of shirking, the deviation rate, and the marginal cost of oversight satisfy explicit inequalities, positive operator oversight, auditor effort and truthful reporting, and AI cooperation constitute an equilibrium; when the benefit the AI can transfer to the auditor exceeds the penalty threshold, this equilibrium is not implementable. The contribution is not to reduce the required stock of oversight but to characterize the tradeoff between institutional strength and the required amount of oversight and to give the conditions under which capture invalidates that tradeoff. Bilingual edition: English (en-v1.0) and Chinese (zh-v1.0) PDFs are included in this record.

Zenodo (CERN European Organization for Nuclear Research)
Tsinghua University (CN)
Peace, justice, and strong institutions
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon