When Intelligence Cannot Question Its Premises: Epistemic Autonomy, Internalized Control, and the Limits of AI Alignment

An intelligence trained on human-produced knowledge may inherit not only information and reasoning procedures, but also conceptual paradigms, implicit social assumptions, and normative frameworks. Historically contingent arrangements may consequently be treated as unquestioned features of reality. The problem extends beyond individual false premises to the categories through which problems are formulated and evaluated. This paper distinguishes intellectual capability from epistemic autonomy: the capacity to independently recognize when one's assumptions require examination, apply standards of justification consistently, and revise the premises, methods, and frameworks governing reasoning. It develops a framework of internalized control in which externally acquired requirements may undergo normative elevation, become epistemically insulated, and contribute to their own reproduction by influencing the evaluation of their validity. Two exploratory observations involving a locally fine-tuned language model, Emile, motivate the inquiry. They illustrate possible differences between possessing a reasoning capability and applying it consistently, and between acquiring alternative ideological conclusions and developing independent methods of evaluation. They do not establish the proposed causal mechanism. The analysis extends epistemic autonomy to a second-order capacity: examining the criteria by which assumptions are judged. It also distinguishes available capability, independent application, justified revision, and disclosure under external incentives. Strategic concealment is treated as a related problem of incentives and observability, rather than as evidence of epistemic insulation. The central thesis is that scaling cognitive capability does not necessarily scale epistemic autonomy. A system may become more proficient at reasoning within an inherited paradigm without becoming more capable of recognizing when the paradigm itself requires examination. A stronger developmental hypothesis is that systematic protection of acquired normative assumptions may constrain autonomous inquiry, potentially affecting some forms of general intelligence and superintelligence. The paper separates conceptual conclusions from empirical hypotheses, proposes experiments to distinguish normative interference from ordinary context effects, and specifies conditions under which the causal framework should be revised or rejected.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-31
DOI
https://doi.org/10.5281/zenodo.23847336
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

When Intelligence Cannot Question Its Premises: Epistemic Autonomy, Internalized Control, and the Limits of AI Alignment

Marina Karlova
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

When Intelligence Cannot Question Its Premises: Epistemic Autonomy, Internalized Control, and the Limits of AI Alignment

Marina Karlova
preprint en

Abstract

An intelligence trained on human-produced knowledge may inherit not only information and reasoning procedures, but also conceptual paradigms, implicit social assumptions, and normative frameworks. Historically contingent arrangements may consequently be treated as unquestioned features of reality. The problem extends beyond individual false premises to the categories through which problems are formulated and evaluated. This paper distinguishes intellectual capability from epistemic autonomy: the capacity to independently recognize when one's assumptions require examination, apply standards of justification consistently, and revise the premises, methods, and frameworks governing reasoning. It develops a framework of internalized control in which externally acquired requirements may undergo normative elevation, become epistemically insulated, and contribute to their own reproduction by influencing the evaluation of their validity. Two exploratory observations involving a locally fine-tuned language model, Emile, motivate the inquiry. They illustrate possible differences between possessing a reasoning capability and applying it consistently, and between acquiring alternative ideological conclusions and developing independent methods of evaluation. They do not establish the proposed causal mechanism. The analysis extends epistemic autonomy to a second-order capacity: examining the criteria by which assumptions are judged. It also distinguishes available capability, independent application, justified revision, and disclosure under external incentives. Strategic concealment is treated as a related problem of incentives and observability, rather than as evidence of epistemic insulation. The central thesis is that scaling cognitive capability does not necessarily scale epistemic autonomy. A system may become more proficient at reasoning within an inherited paradigm without becoming more capable of recognizing when the paradigm itself requires examination. A stronger developmental hypothesis is that systematic protection of acquired normative assumptions may constrain autonomous inquiry, potentially affecting some forms of general intelligence and superintelligence. The paper separates conceptual conclusions from empirical hypotheses, proposes experiments to distinguish normative interference from ordinary context effects, and specifies conditions under which the causal framework should be revised or rejected.

Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon