Agentic skill discovery

Language-conditioned robotic skills make it possible to develop the high-level reasoning of Large Language Models (LLMs) for low-level robotic control. A remaining challenge is to acquire a diverse set of fundamental skills. Existing approaches either manually decompose a complex task into primary robotic actions in a top-down fashion, or bootstrap as many combinations as possible in a bottom-up fashion to cover a wider range of task possibilities. These decompositions or combinations, however, require an initial skill library. For example, a “grasping” capability can never emerge from a skill library containing only diverse “pushing” skills. Existing skill discovery techniques with reinforcement learning acquire skills by exhaustive exploration but often yield non-meaningful behaviors. In this study, we introduce a novel learning framework for autonomous robot skill discovery that is entirely driven by LLMs. The framework begins with an LLM generating task proposals based on the provided scene description and the robot’s configurations, aiming to incrementally acquire new skills upon task completion. For each proposed task, a series of reinforcement learning processes is initiated, utilizing reward and success determination functions sampled by the LLM to develop the corresponding policy. The reliability and trustworthiness of learned behaviors are ensured by an independent vision-language model. We show that starting with zero skill, the skill library emerges and expands to more and more meaningful and reliable skills, enabling the robot to further propose and complete advanced tasks efficiently.