Visibility-Aware Task and Motion Planning with Task-Centric Perception
Long-horizon manipulation under uncertainty remains a challenging problem in robotics. We present \ourmethod{}, an algorithm for Visibility-Aware Task and Motion Planning. \ourmethod{} extends existing deterministic task and motion planning solvers to partially observable object rearrangement tasks, while still being able to take advantage of the efficiency and compositional generalization over long horizons of the base solver. This extension interleaves sensing actions into a plan skeleton that assumed full observability, and uses a novel persistent object-based memory to fuse observations into object mesh hypotheses and an observed occupancy map. We evaluate \ourmethod{} on real-world tabletop rearrangement tasks which feature heavy occlusion and partial visibility and demonstrate that it outperforms baselines that do not perform active perception or have persistent memory.
Authors
- Leslie Pack Kaelbling (ORCID: https://orcid.org/0000-0001-6054-7145)
- Aryan Zoroufi (ORCID: https://orcid.org/0000-0003-2481-6883)
- Sahit Chintalapudi
- Tomás Lozano-Pérez
- Aditya Agarwal
- Xiaoyan Wang
- Nancy Kanwisher
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.48550/arxiv.2611.16213
- Primary Topic
- Robot Manipulation and Learning
- Type
- preprint