Purpose-driven Open-Ended Learning: biasing OEL through external guidance

Open-ended learning (OEL) enables autonomous agents to discover and practice new skills in a self-motivated fashion, but it does not inherently guarantee alignment with user-defined needs. In real-world scenarios, this gap can lead the robot to invest excessive effort in acquiring knowledge that is irrelevant to the user’s broader objectives. To address this issue, we introduce a purpose-driven OEL framework that combines intrinsic exploration with an extrinsic motivational channel reflecting the user’s overarching objectives (the purpose). Our approach augments H-GRAIL, an existing hierarchical OEL architecture, with a dedicated purpose-oriented bias. During OEL learning, this bias nudges the agent to favour goals that share features with previously rewarded states, accelerating competence acquisition on user-relevant tasks. The agent is also encouraged to explore goals similar to those that previously fulfilled the user’s request, even if they haven’t produced any reward yet. We validate our method in a 2D grid-world and in a real robotic scenario, both featuring multiple domains. Results show that the proposed solution improves the agent’s ability to collect user-relevant rewards compared to a an intrinsic baseline with no purpose-related bias, especially in environments containing many distracting goals, while preserving the agent’s capacity to discover and learn about the environment autonomously.