Demonstration to Adaptation: A User-Guided Framework for Sequential and Real-Time Planning

This paper introduces a comprehensive user-guided planning framework designed for robots operating in dynamic, human-centered environments – where the ability to execute sequential tasks flexibly and adaptively is paramount. Our planner enables robots to (i) encode object-centric constraints and user preferences via multiple demonstrations, (ii) transfer geometric features and implicit relaxations to novel scenarios while reacting to unforeseen events, and (iii) adapt to changing task conditions in real-time, including the real-time replanning and tracking of moving targets. Our approach relies on C<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> screw linear interpolation, which generates smooth paths satisfying the underlying geometric constraints that characterize the task. The prescribed path is combined with a hierarchical quadratic programming-based controller which explores the user demonstrations's stochastic variability to relax task constraints while ensuring real-time whole-body collision avoidance. Our framework continuously checks for dynamic changes in task targets, ensuring appropriate planning or control actions, and tending to the prescribed screw path. This comprehensive approach is deployed in different task conditions which are available at https://youtu.be/F0cMr1n1D9k.