JEPA as a Latent World Model for Model-Predictive Control in Simulated Physics and Language-Specified Goals

A world model predicts how the environment evolves, which allows an agent to plan without acting. We study a joint-embedding predictive architecture (JEPA) that predicts future latent states conditioned on actions, and use it inside a model-predictive controller (MPC). We evaluate planning success in simulated control tasks and check whether the latent space preserves the dynamics using known simulator states. Goals are specified in natural language and mapped to target latents through a frozen text encoder. A pilot arm with a linear model saturates at R^2 0.998, so the JEPA results are pending. Preprint. Results in this version come from small-scale pilot experiments; the full experimental protocol is described in the text and is pending.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-02
DOI
https://doi.org/10.5281/zenodo.23890646
Primary Topic
Reinforcement Learning in Robotics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

JEPA as a Latent World Model for Model-Predictive Control in Simulated Physics and Language-Specified Goals

Dheiver Francisco Santos
Zenodo (CERN European Organization for Nuclear Research)
Reinforcement Learning in Robotics
preprint

JEPA as a Latent World Model for Model-Predictive Control in Simulated Physics and Language-Specified Goals

Dheiver Francisco Santos
preprint en

Abstract

A world model predicts how the environment evolves, which allows an agent to plan without acting. We study a joint-embedding predictive architecture (JEPA) that predicts future latent states conditioned on actions, and use it inside a model-predictive controller (MPC). We evaluate planning success in simulated control tasks and check whether the latent space preserves the dynamics using known simulator states. Goals are specified in natural language and mapped to target latents through a frozen text encoder. A pilot arm with a linear model saturates at R^2 0.998, so the JEPA results are pending. Preprint. Results in this version come from small-scale pilot experiments; the full experimental protocol is described in the text and is pending.

Zenodo (CERN European Organization for Nuclear Research)
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon