Developing an automatic gripping learning system for a robotic arm by integrating a convolutional neural network and optimization algorithms

• This study used optimization algorithms to automatically detect the optimal positions of a robotic arm for object gripping in a simulation environment and used the data thus obtained as training data to reduce the difficulty of and time required for real-world data collection. • This study designed a gripping system that is low in cost given that it does not require expensive sensors. • This study designed a method to convert coordinates and combined this method with YOLO to precisely locate objects, thereby greatly reducing the impact of distortion. This study used a convolutional neural network (CNN) and optimization algorithms to develop an automatic gripping system for a robotic arm, overcoming the limitations of traditional learning methods. Unlike existing approaches that require retraining and parameter adjustments for various scenarios, this research demonstrates more flexible and accurate performance. Subsequently, this study built an experimental setting in a physical simulator so that multiple optimization algorithms could be constructed and object detection algorithms could be trained to extract required data. Since the real-world environment inevitably differed slightly from the simulation environment, this study extracted a small amount of data from the real-world environment to perform transfer learning. Finally, this study applied the gripping system to a robotic arm to verify its performance. The experimental results revealed that the deep learning model developed in this study had 96 % success rate for gripping objects. The mean absolute error was 4.06, the root mean squared error was 5.03, and the R 2 was 0.99. Optimization algorithms were used in the simulation environment to collect training data, which in turn enabled the system to automatically find the optimal posture for gripping objects. When the object to be gripped changed, the automatic gripping system could automatically retune itself accordingly; thus, it did not require manual tuning. Finally, the automatic gripping system was able to control a robotic arm for the automatic gripping of objects. This system is suitable for application in industry to enhance the performance of production lines and render the arrangement of production lines more flexible.