A Model-Class Comparison of Large Language Models for Translating Chinese Route Instructions into Robot Command Sequences
Advances in large language models (LLMs) have accelerated the real-world deployment of embodied intelligent robots, supporting intuitive human–robot interaction without specialized robotics expertise. However, few systematic assessments examine how well LLMs translate natural-language route instructions into executable robot command sequences, particularly for non-English instructions and across deployment-relevant model classes. In this paper, a Chinese route-instruction corpus was constructed, which comprised 212 instructions collected from six native speakers across 50 procedurally generated town maps. Each was annotated with a ground-truth command sequence verified by simulation to reach its goal, and a 100% performance ceiling was established. Using this corpus, 12 LLMs spanning three capability tiers—cloud proprietary models, mid-sized open-weight models, and small open-weight models—were evaluated under three system prompts differing only in the number of worked examples (zero, one, and three). Open-weight models in the 3–9B range reached at most 9.2% of goals, while proprietary cloud models reached 80.0%, a substantial gap that increasing the number of worked examples did not measurably narrow. A paired analysis across the three prompt conditions detected no reliable effect of demonstration count: pooled path_reach rose modestly from 25.0% under zero-shot prompting to 27.2% with one example and 27.7% with three examples (three versus zero examples: odds ratio: 1.15; 95% CI: 0.98–1.34). None of these differences was significant after multiple-comparison correction (Holm-adjusted p ≥ 0.247). Low-Rank Adaptation (LoRA) fine-tuning on 126 examples raised mid-sized open-weight models to 27.7–35.4%, and the gain spanned both output-format compliance and route competence.
Authors
- Ruizhe Yang (ORCID: https://orcid.org/0000-0003-0582-4079)
- Yuki Nishimura (ORCID: https://orcid.org/0009-0004-7923-5036)
Institutions
- Hangzhou Dianzi University (CN)
Publication Details
- Journal
- Journal of Sensor and Actuator Networks
- Published
- 2026-11-17
- DOI
- https://doi.org/10.3390/jsan15060091
- Primary Topic
- Speech and dialogue systems
- Type
- article
- Field-Weighted Citation Impact
- 0.00