Breaking the Code Barrier: Analyzing the Effectiveness of LLMs in Natural Language Interfaces for Collaborative Robotics

The integration of natural language interfaces in collaborative robotics presents a novel approach to simplifying human-machine interaction. This research investigates the challenge of enabling intuitive, non-programmatic control of a standard collaborative robot using Large Language Models (LLMs) to interpret and execute user commands. Despite advancements in LLMs, existing solutions are not yet able to fully address the variability in performance across different models, particularly in terms of robustness, reliability, and situational behavior. To explore this gap, we implemented a system that allows users to control the robot via natural language commands, delivered through Microsoft HoloLens 2 or a tablet PC interface, and tested the system using four different LLMs (two versions of Chat-GPT and two versions of Llama). Our study reveals significant differences in the models' capabilities, with GPT-4o achieving the highest accuracy but still exhibiting some limitations. These findings highlight the potential of LLMs in revolutionizing human-machine interaction, while also pointing to the need for further improvements to enhance reliability and safety. This work contributes to the ongoing development of more accessible and robust collaborative robotic systems, paving the way for broader adoption in various industries.