Design of a Service Robot System Based on a Multimodal Large Model

In recent years, significant progress has been made in the fields of vision, language, and robotics, driving the development of generalist robot and bringing them closer to people's daily lives. Facing complex unstructured environments, robots need to possess a series of key capabilities, including scene building, mobile navigation, task execution, scene understanding and task planning. In this paper, a service robot system based on a multimodal large model is proposed. The main elements of the system cover the hardware system design of the service robot, the development of scene building algorithms, the mobile navigation strategy based on the TEB (Timed Elastic Band) path planning algorithm, the robot grasping based on the pre-trained model, the fine task manipulation based on learning, and the integrated solution for task planning relying on the visual-linguistic model. To evaluate the system performance, several tasks were designed in a laboratory scenario and several experiments were conducted. The results show that the system performs well in scene building, mobile navigation and robot static scene manipulation, but the overall performance still needs to be further improved. Finally, prospects for future work include enhancing simulations and incorporating force sensors.