Interactive Conversational AI with IoT Devices for Enhanced Human-Robot Interaction

Significance - The rapid advancements in conversational AI and IoT technologies have opened up new possibilities for human-machine interaction. Despite the progress, a gap exists in integrating these two fields to create more centralized, intuitive, and engaging user experiences. Current integrations typically consist of specialised hardware-software pairs that do not fully leverage the capabilities of advanced conversational models, thereby limiting their applicability. This research proposes a general solution to bridge the capabilities of various IoT devices with the oversight and control abilities of AI language models, enhancing the potential for more versatile and natural IoT-AI-human interactions. Aim and Approach - This research presents the design and development of an IoT system operated by an AI language model and conversationally managed by humans to operate robots. Based on this setup, the initial goal is to create a framework for interactively controlling a robotic arm. The approach involves using a Raspberry Pi as a central control system and ChatGPT API to manage conversations and execute given commands. Results - The developed IoT-AI system demonstrated efficient and reliable human-robot interaction. It effectively captures user voice inputs, processes them through advanced AI models, and generates appropriate commands for the robotic arm, achieving an average voice-to-motion latency of 5.5 seconds. While some latency and voice recognition challenges exist, the overall performance confirms the viability of using conversational AI for natural and intuitive robotic control. Conclusions - This research successfully integrates conversational AI with IoT devices, resulting in a more user-centric and efficient human-robot interaction. The system highlights the significant potential of precisely translating natural language commands into robotic actions, enhancing user experience and operational efficiency.