Functional Self-Modeling Without Phenomenal Consciousness: Architectural Conditions and Challenges for Self-Aware Agency in AI Systems with Environment Interfaces

This position paper advances a conceptual analysis of functional self-modeling---a form of self-aware agency distinct from phenomenal consciousness---drawing on insights from global workspace theory (GWT), predictive processing (PP), and embodied sensorimotor theory. As a position paper, we do not present a validated theory or an implemented system; rather, we articulate a structured conceptual framework: four architectural conditions proposed as one coherent design pathway for realizing functional self-modeling (subject to empirical refinement), their interactions, and the open problems they raise---designed to stimulate and guide future work on self-modeling in AI. We explicitly bracket the "hard problem" of phenomenal consciousness and focus on the architectural and computational features that enable a system to model itself as a bounded, temporally persistent causal agent. We identify four architectural conditions for functional self-modeling: self-world distinction, causal action-feedback closure, temporal continuity, and autonomous self-update---a proposed translation of the functional selfhood definition (the target capacity, Section 2.2) into concrete architectural conditions (one realization pathway, Section 4). While these conditions apply most directly to AI systems with external environment interfaces (robotic, simulator-API, human-in-the-loop), we discuss their implications for extended language-based architectures. We explore, as a speculative direction, whether these conditions may amplify instrumental convergence risks, illustrated via a self-modeling variant of Bostrom's paperclip maximizer, and discuss internal competition among subsystems---abstracted from the dynamic rivalry observed in biological cognitive architectures---as a conceptual direction for constraint design. We critically engage with existing architectures (MemGPT, Voyager) and philosophical debates (Chalmers 2023, Seth 2018, Shevlin 2021), while acknowledging significant unresolved challenges: no empirical validation is provided; the conditions are a design proposal, not a set of proven necessary conditions; behavioral tests remain to be developed; and the compatibility of autonomous self-modification with fixed safety constraints is an open problem. This work offers not a completed framework but a set of conceptual foundations and open questions for the study of self-modeling in artificial agents. By specifying architectural conditions for self-modeling agency independently of phenomenal consciousness, the framework also provides a functional baseline for consciousness research: it identifies computational prerequisites whose relationship to phenomenal experience remains an open empirical question, and against which claims about machine consciousness can be systematically evaluated.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-27
DOI
https://doi.org/10.5281/zenodo.20964200
Primary Topic
Embodied and Extended Cognition
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Functional Self-Modeling Without Phenomenal Consciousness: Architectural Conditions and Challenges for Self-Aware Agency in AI Systems with Environment Interfaces

Yu Hu
Zenodo (CERN European Organization for Nuclear Research)
Embodied and Extended Cognition
preprint

Functional Self-Modeling Without Phenomenal Consciousness: Architectural Conditions and Challenges for Self-Aware Agency in AI Systems with Environment Interfaces

Yu Hu
preprint en

Abstract

This position paper advances a conceptual analysis of functional self-modeling---a form of self-aware agency distinct from phenomenal consciousness---drawing on insights from global workspace theory (GWT), predictive processing (PP), and embodied sensorimotor theory. As a position paper, we do not present a validated theory or an implemented system; rather, we articulate a structured conceptual framework: four architectural conditions proposed as one coherent design pathway for realizing functional self-modeling (subject to empirical refinement), their interactions, and the open problems they raise---designed to stimulate and guide future work on self-modeling in AI. We explicitly bracket the "hard problem" of phenomenal consciousness and focus on the architectural and computational features that enable a system to model itself as a bounded, temporally persistent causal agent. We identify four architectural conditions for functional self-modeling: self-world distinction, causal action-feedback closure, temporal continuity, and autonomous self-update---a proposed translation of the functional selfhood definition (the target capacity, Section 2.2) into concrete architectural conditions (one realization pathway, Section 4). While these conditions apply most directly to AI systems with external environment interfaces (robotic, simulator-API, human-in-the-loop), we discuss their implications for extended language-based architectures. We explore, as a speculative direction, whether these conditions may amplify instrumental convergence risks, illustrated via a self-modeling variant of Bostrom's paperclip maximizer, and discuss internal competition among subsystems---abstracted from the dynamic rivalry observed in biological cognitive architectures---as a conceptual direction for constraint design. We critically engage with existing architectures (MemGPT, Voyager) and philosophical debates (Chalmers 2023, Seth 2018, Shevlin 2021), while acknowledging significant unresolved challenges: no empirical validation is provided; the conditions are a design proposal, not a set of proven necessary conditions; behavioral tests remain to be developed; and the compatibility of autonomous self-modification with fixed safety constraints is an open problem. This work offers not a completed framework but a set of conceptual foundations and open questions for the study of self-modeling in artificial agents. By specifying architectural conditions for self-modeling agency independently of phenomenal consciousness, the framework also provides a functional baseline for consciousness research: it identifies computational prerequisites whose relationship to phenomenal experience remains an open empirical question, and against which claims about machine consciousness can be systematically evaluated.

Zenodo (CERN European Organization for Nuclear Research)
Embodied and Extended Cognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.