CLFR-M: Continual Learning Framework for Robots via Human Feedback and Dynamic Memory

Robots working in dynamic real-world environments need continual learning to adapt to changing situations and challenges. Traditional robotics learning methods lack robustness and transferability and thus are inefficient for complex open-ended robot tasks in dynamic environments. In recent years, large language models(LLMs) have become one of the most promising schemes for robot planning tasks thanks to their generalization and convenience in different tasks. However, LLMs-planners are not good at continuously learning from long-term experience. To address this problem, we present a continual learning framework for robots using LLMs via human feedback and dynamic memory (CLFR-M), which continuously improves robots’ behavior without additional training or intricate finetuning. We have built a persistent, dynamic embedding memory and practical knowledge structure to record and organize important experiences. Meanwhile, we designed a memory-inspired action generation method to incorporate accumulated experience as semantic guidance to indicate the direction of improvement. Moreover, we introduce human feedback into the iterative process of robot learning to rectify errors and supply specific knowledge. In the Virtual Home benchmark, CLFR-M outperforms classical methods like ProgPrompt. Meanwhile, CLFR-M also can transfer its learned knowledge to unseen tasks, showing good zero-shot task performance.