LAC: Using LLM-based Agents as the Controller to Realize Embodied Robot

The rapid development of Large Language Models (LLMs) facilitates the application of robotics, especially for robotic control. LLM-based robots are a way to realize embodied intelligence, but they still lack a general control paradigm to achieve embodied robot. We propose LAC (LLM-based Agents as the Controller), a new formulation to enable embodied robots to autonomously react to high-level tasks like humans. Specifically, the controller in LAC is LLM-based agents that can plan, decompose tasks and make decisions based on linguistic input. Two agents operate in parallel within the controller to replicate both instinctive reaction and deep consideration. The output of the controller consists of linguistic parameters that activate task-specific tools, enabling autonomous execution of low-level actions. LAC translates multi-modal feedback information into linguistic messages, which are then transmitted back to the controller to establish a closed-loop control flow. Once equipped with appropriate tools, the proposed LAC can be applied across various applications. A straightforward simulated office task, TASK0, illustrates the successful completion of the embodied task by LAC. Furthermore, testing five comparative LLMs on TASK0 also demonstrates that LAC serves as a potential test platform for validating the capability of LLMs to function as the cognitive center of embodied robots.