Preemptive solving of future problems: Multitask Preplay in humans and machines

Humans can pursue a near-infinite variety of tasks, but typically can only pursue a small number at the same time. We hypothesize that humans leverage experience on one task to preemptively learn solutions to other tasks that were accessible but not pursued. We formalize this idea as “Multitask Preplay”, an algorithm that replays experience on one task as the starting point for “preplay”—counterfactual simulation of an accessible but unpursued task. Preplay is used to learn a predictive representation that can support fast, adaptive task performance later on. When implemented within deep reinforcement learning, it predicts human generalization to novel tasks more accurately than traditional planning and predictive representation methods across grid-worlds and a 2D Minecraft domain. Additionally, Multitask Preplay improves an artificial agent’s ability to generalize to new environments sharing task co-occurrence structure. These findings demonstrate that Multitask Preplay is a scalable theory of human counterfactual learning and generalization; endowing artificial agents with this capacity can significantly improve their performance in challenging multitask environments.

Authors

Institutions

Publication Details

Journal
Proceedings of the National Academy of Sciences
Published
2026-11-19
DOI
https://doi.org/10.1073/pnas.2521870123
Citations
3
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
0.97

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Preemptive solving of future problems: Multitask Preplay in humans and machines

Samuel Joseph Gershman, Sam Hall-McMaster, Wilka Carvalho, Honglak Lee
3 citations
Proceedings of the National Academy of Sciences
Reinforcement Learning in Robotics
0.97
article

Preemptive solving of future problems: Multitask Preplay in humans and machines

Samuel Joseph Gershman, Sam Hall-McMaster, Wilka Carvalho, Honglak Lee
article en
3 citations

Abstract

Humans can pursue a near-infinite variety of tasks, but typically can only pursue a small number at the same time. We hypothesize that humans leverage experience on one task to preemptively learn solutions to other tasks that were accessible but not pursued. We formalize this idea as “Multitask Preplay”, an algorithm that replays experience on one task as the starting point for “preplay”—counterfactual simulation of an accessible but unpursued task. Preplay is used to learn a predictive representation that can support fast, adaptive task performance later on. When implemented within deep reinforcement learning, it predicts human generalization to novel tasks more accurately than traditional planning and predictive representation methods across grid-worlds and a 2D Minecraft domain. Additionally, Multitask Preplay improves an artificial agent’s ability to generalize to new environments sharing task co-occurrence structure. These findings demonstrate that Multitask Preplay is a scalable theory of human counterfactual learning and generalization; endowing artificial agents with this capacity can significantly improve their performance in challenging multitask environments.

Proceedings of the National Academy of SciencesVol. 123(47)
Harvard University (US), University of Michigan (US), LG AI Research (South Korea) (KR)
U.S. Department of Defense, Multidisciplinary University Research Initiative, Army Research Office
Openalex Percentile: Top 23%
Reinforcement Learning in Robotics
0.97
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon