Human-to-Robot Imitation with Symbolic Planning in a Unified Latent Space

Direct imitation of humans by robots offers a promising direction for remote teleoperation and intuitive task instruction, where a human can perform a task naturally and the robot autonomously interprets and executes it using its own embodiment. Existing methods often rely on close alignment between human and robot scenes. This prevents robots from inferring the intent of the task or executing demonstrated behaviors when the initial states mismatch. Hence, it poses difficulties for non-expert users, who may need domain knowledge to adjust the setup.