LLM-Based Task Planning for Mobile Robots Using a Novel SLAM Method for Materials Distribution

This paper presents a novel autonomous mobile robot system for material distribution, integrating dynamic map generation, natural language processing (NLP), and efficient path-planning for real-world factory applications. The system begins with a top-view image capture of the environment using a fisheye-lens camera, followed by image correction and color-based object detection to identify stations and generate an occupancy map for navigation, eliminating the need for traditional SLAM techniques. User interaction is facilitated through an NLP module connected to a large language model (LLM) trained on predefined assembly manuals, enabling the robot to interpret requests and determine the required materials for specific assembly tasks. The global path-planning is managed by the Improved RRT* with Reduced Random Map Size (IRRT*-RRMS) algorithm, ensuring optimal and efficient motion in dynamic environments. Experimental validation using a TurtleBot3 Waffle Pi equipped with an OpenManipulator-X demonstrates the system's effectiveness, particularly in handling assembly model H, which includes bolts, nuts, and screws.