Goal-Conditioned Self-Supervised Learning from Play Aimed at Human–Robot Collaboration

Robots capable of accurately understanding situations based on environmental information and collaborating effectively with humans in a variety of tasks are in high demand. Deep-learning-based approaches can be effective for realizing such collaborative robots; however, collecting data and achieving high generalization performance are challenges. To address this issue, we utilize “play data” collected by teleoperating a robot when performing collaborative tasks with a human partner. This study proposes a play-supervised latent goal for human–robot collaboration (play-HRC) framework that learns from unlabeled play data for human–robot collaborative scenarios without explicit goal observation. Unlike play-supervised latent motor plan that generates actions conditioned on an externally specified goal observation, play-HRC must infer a latent goal from an initial state. At the beginning of the task execution, play-HRC infers an achievable goal from its initial state. During execution, it generates actions and predicts future environmental states while adaptively optimizing the latent goal by minimizing prediction errors. To evaluate the performance of play-HRC, we conducted tests in a real-world environment using a task in which a robot and human collaboratively manipulate objects. The results highlighted the ability of the framework to infer achievable goals, generate actions, and dynamically adjust goals in response to human behavior.