Advanced Human-Robot Interaction with a Mobile Robot: Integrating Large Language Models into Robotics

Advancements in assistive robotic systems demand sophisticated Human-Robot Interaction (HRI) strategies, particularly in multilingual environments. This study introduces an HRI framework that integrates Natural Language Processing (NLP) via two Large Language Models (LLMs) accessed through backend applications interfacing with OpenAI and Deepgram APIs. The system employs a responsive chat-style interface that is usable across multiple devices, supporting Brazilian Portuguese, a notable improvement over similar projects. An experimental protocol was conducted with 20 native Portuguese-speaking volunteers to evaluate the system’s interpretive performance and user perception. Participants guided a mobile robot using voice commands to complete a predefined trajectory in an obstacle-free environment. Usability evaluations using the System Usability Scale yielded consistently high scores throughout the experiment, ranging from “good” to “excellent”. However, users noted slightly reduced accuracy perception in fully autonomous LLM mode compared to a pre-programmed mode, alongside increased frustration. These findings validate the feasibility of integrating LLMs into multilingual robotic systems, emphasizing both the promise and limitations of NLP in HRI. Future work could focus on improving the handling of ambiguous user requests and enhance feedback mechanisms to improve user experience.