Preemptive solving of future problems: Multitask Preplay in humans and machines
Humans can pursue a near-infinite variety of tasks, but typically can only pursue a small number at the same time. We hypothesize that humans leverage experience on one task to preemptively learn solutions to other tasks that were accessible but not pursued. We formalize this idea as “Multitask Preplay”, an algorithm that replays experience on one task as the starting point for “preplay”—counterfactual simulation of an accessible but unpursued task. Preplay is used to learn a predictive representation that can support fast, adaptive task performance later on. When implemented within deep reinforcement learning, it predicts human generalization to novel tasks more accurately than traditional planning and predictive representation methods across grid-worlds and a 2D Minecraft domain. Additionally, Multitask Preplay improves an artificial agent’s ability to generalize to new environments sharing task co-occurrence structure. These findings demonstrate that Multitask Preplay is a scalable theory of human counterfactual learning and generalization; endowing artificial agents with this capacity can significantly improve their performance in challenging multitask environments.
Authors
- Samuel Joseph Gershman (ORCID: https://orcid.org/0000-0002-6546-3298)
- Sam Hall-McMaster (ORCID: https://orcid.org/0000-0003-1641-979X)
- Wilka Carvalho (ORCID: https://orcid.org/0000-0001-9350-9751)
- Honglak Lee (ORCID: https://orcid.org/0000-0002-4109-327X)
Institutions
- Harvard University (US)
- University of Michigan (US)
- LG AI Research (South Korea) (KR)
Publication Details
- Journal
- Proceedings of the National Academy of Sciences
- Published
- 2026-11-19
- DOI
- https://doi.org/10.1073/pnas.2521870123
- Citations
- 3
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- article
- Field-Weighted Citation Impact
- 0.97
Funders
- U.S. Department of Defense
- Multidisciplinary University Research Initiative
- Army Research Office