Development of a Digital Twin for Robotic Inspection Using Computer Vision and Machine Learning

This article reveals the creation and execution of a digital twin (DT) for the robotic inspection system of mechanical parts, applying the machine vision method. The examination is carried out on the KUKA iiwa robot that has a camera for the purpose of identifying and sorting the parts that are positioned on the work area. A vision system based on a YOLO convolutional neural network (CNN) was utilized for the tasks of object detection and classification. The dataset was automatically created via the use of 3D CAD models along with domain randomization to mitigate the risk of overfitting. The simulation environment was set up in IsaacSim, and the vision system was interfaced through ROS Noetic with hand-eye calibration also included. The performance of the YOLO network evidenced an overall precision of 94.8% and recall of 87.7% in the test dataset indicating the parts' effective detection and classification. The research underlines the convergence of simulation, deep learning, robotics, and the potential for automated inspections in manufacturing sectors.