HandWorldBench: A Unified Benchmark for Hand-Centric World Models
Recent advances in world models are opening new possibilities for fine-grained embodied tasks, with hand-object interaction offering a particularly demanding setting. Modeling these interactions involves capturing how hand actions change objects and how the resulting states persist over time. Yet systematic evaluation tailored to these hand-centric capabilities remains underdeveloped. To address this gap, we introduce HandWorldBench, a unified benchmark that systematically evaluates hand-centric world models across human hands, simulated dexterous hands, and real dexterous hands along six dimensions: visual quality, hand fidelity, appearance consistency, instruction following, physical consistency, and temporal coherence. To construct the benchmark, we develop a data curation pipeline that extracts hand-object interaction clips and screens them for quality, yielding 20,500 high-quality egocentric clips from over 600 hours of footage. We then manually select 500 clips covering seven task families for evaluation. Benchmarking eleven models under a unified protocol reveals that stronger hand-pose control does not consistently translate into coherent object-state evolution and persistence or robust cross-embodiment performance, with several specialized hand-control models underperforming general-purpose text/image-to-video generators on dexterous-hand tasks.
Authors
- Xvyuan Liu
- Shuqing Wang (ORCID: https://orcid.org/0000-0002-1092-8166)
- Zhaolu Wang
- Rongze Tang
- Xin Zhang
- Yong Li
- Jianjie Fang
- Lihong Wang
- Jiacheng Xu
- Ziyou Wang
- Jiawei Chen
- Wei Wu
- Yiding Ma
- Chen Gao
- Zhibo Chen
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.48550/arxiv.2611.24731
- Primary Topic
- Robot Manipulation and Learning
- Type
- preprint