OFSLMs: Offline Fine-Tuned Small Language Models Based on Hybrid Synthetic Knowledge for Robot Introspective Decision-Making

The combination of robots with large language models (LLMs) offers the potential to create more intelligent systems, a prospect that has garnered attention from researchers in recent years. Current efforts primarily focus on constructing prompt systems through online APIs and high-level natural language-based planning and decision-making offline. However, these approaches often neglect scenarios with weak network connectivity and downplay the close integration between models and the robot's lower-level control. We propose that robots should be equipped with Offline Fine-tuned Small Language Models (OFSLMs) capable of generating introspective decisions offline. To this end, we design a workflow for creating these OFSLMs and develop a hybrid synthetic fine-tuning dataset with human and GPT-4's annotation. This dataset contains 2,663 cases to train and evaluate OFSLMs to generate “plan” and lower-level control codes called “solution” based on given “instruction.” Additionally, we devise a two-stage evaluation method to select the best OFSLMs. In experiments, we analyze the distribution of the dataset, and then we select the recent 9 open-source small language models and filter out the best 3 models using proposed workflow. We further perform nonlinear fitting on these OFSLMs' numerical metrics, and explore the generalization capabilities of the OFSLMs. Experiments demonstrate that the dataset we create is semantically rich and that our proposed workflow yields high-quality OFSLMs. These models exhibit non-linear growth in metrics. Moreover, our OFSLMs demonstrate the ability to perform offline robot decision-making and generate control code, highlighting their specialization; simultaneously, OFSLMs also show comparable performance in non-decision-related text-based question-answering tasks, indicating their generalization capability.