Assistive Robot for Automatic Sorting Using Voice-Guided Human-Machine Interaction

This paper presents the development of a virtual environment for object classification and location by an assistive robot using voice-guided human-machine interaction to facilitate the storage or ordering of products by means of robotic systems, reducing potential risks due to excess load or repeatability to human operators. The proposed approach integrates computer vision techniques and transformer-based speech recognition models within a virtual simulation environment. Specifically, a ResNet18 neural network, selected for its low computational demand and high efficiency in classification and localization tasks, is used to accurately identify objects. As a contribution to the state of the art, a human-machine interaction environment is developed with natural language processing algorithms oriented toward industrial applications, where the sorting order is specified by voice commands captured and transcribed by a wav2vec-based speech-to-text algorithm, allowing users to interact naturally and efficiently with the robotic system. Experimental validation demonstrates the robustness of object detection and the reliability of speech recognition, highlighting the system’s effectiveness and potential applications in automated industrial scenarios. Keywords: ResNet, Speech to text (STT), Human Robot collaboration, Automation.