Gradient-Free De Novo Learning

This technical note applies active inference to the problem of learning goal-directed behaviour from scratch, namely, de novo learning. By de novo learning, we mean discovering, directly from observations, the structure and parameters of a discrete generative model for sequential policy optimisation. Concretely, our procedure grows and then reduces a model until it discovers a pullback attractor over (generalised) states; this attracting set supplies paths of least action among goal states while avoiding costly states. The implicit efficiency rests upon reframing the learning problem through the lens of the free energy principle, under which it is sufficient to learn a generative model whose dynamics feature such an attracting set. For context, we briefly relate this perspective to value-based formulations (e.g., Bellman optimality) and then apply the active inference formulation to a small arcade game to illustrate de novo structure learning and ensuing agency.

Authors

Institutions

Publication Details

Journal
Entropy
Published
2025-09-22
DOI
https://doi.org/10.3390/e27090992
Citations
1
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
2.48

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Gradient-Free De Novo Learning

Thomas Parr, Conor Heins, Alexander Tschantz, Tommaso Salvatori et al.
1 citations
Entropy
Reinforcement Learning in Robotics
2.48
article

Gradient-Free De Novo Learning

Thomas Parr, Conor Heins, Alexander Tschantz, Tommaso Salvatori, Magnus Koudahl, Toon Van de Maele, Christopher Dominic Buckley, Tim Verbelen, Karl Friston, Lancelot Da Costa
article en
1 citations

Abstract

This technical note applies active inference to the problem of learning goal-directed behaviour from scratch, namely, de novo learning. By de novo learning, we mean discovering, directly from observations, the structure and parameters of a discrete generative model for sequential policy optimisation. Concretely, our procedure grows and then reduces a model until it discovers a pullback attractor over (generalised) states; this attracting set supplies paths of least action among goal states while avoiding costly states. The implicit efficiency rests upon reframing the learning problem through the lens of the free energy principle, under which it is sufficient to learn a generative model whose dynamics feature such an attracting set. For context, we briefly relate this perspective to value-based formulations (e.g., Bellman optimality) and then apply the active inference formulation to a small arcade game to illustrate de novo structure learning and ensuing agency.

EntropyVol. 27(9)
University of Sussex (GB), Queen Mary University of London (GB), University of Oxford (GB), Bernstein Center for Computational Neuroscience Tübingen (DE), VERSES (United States), University College London (GB), University of Tübingen (DE)
National Institute for Health and Care Research
Openalex Percentile: Top 8%
Reinforcement Learning in Robotics
2.48
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.