CORA: A Chain of Robotic Actions Reasoning Model for Autonomous Robotic Arm Manipulation

The integration of large language models, combined with pretrained skills, has emerged as a powerful tool for proposing natural language actions that are both feasible and contextually appropriate to perform real-world tasks. In this paper, to further enhance both the feasibility and contextual appropriateness of actions within physical environments, we introduce CORA (Chain Of Robotic Actions), a model that leverages the chain of thought methodology within multimodal foundation models for robust robotic arm manipulation. CORA initiates the task-grounding process through the chain of thought approach, adeptly identifying action chains from high-level hu-man instructions. This method decomposes complex tasks into transparent, intermediate steps, boosting the model's precision and clarity in problem-solving. The chain of thought methodology not only provides a granular and explicit breakdown of the task-solving process but also improves the model's capacity to address complex, multi-step, and implicit instructions with enhanced interpretability. In the subsequent phase, CORA utilizes a multimodal foundation model for world-grounding, assisting the robot in completing the action plan within the physical environment. In a rigorous experimental setup, we evaluate our method in multiple contexts, including simulated and real-world environments. Ablation studies are conducted to evaluate the effectiveness of the Chain of Robotic Action Reasoning strategies in comprehending action chains explicitly and implicitly stated in human instructions. A comparative analysis with existing approaches further substantiates its superior performance. The results conclusively demonstrate the method's effectiveness and efficiency across various components.