Predicting Maximum Permitted Process Forces for Object Grasping and Manipulation Using a Deep Learning Regression Model

During the execution of handling processes in manufacturing, it is difficult to measure the process forces with state-of-the-art gripper systems since they usually lack integrated sensors. Thus, the exact state of the gripped object and the actuating process forces during manipulation and handling are unknown. This paper proposes a deep learning regression model to construct a continuous stability metric to predict the maximum process forces on the gripped objects using high-resolution optical tactile sensors. A pull experiment was developed to obtain a valid dataset for training. Continuously force-based labeled pairs of tactile images for varying grip positions of industrial gearbox parts were acquired to train a novel neural network inspired by encoder-decoder architectures. A ResNet-18 model was used for comparison. Both models can predict the maximum process force for each object with a precision of less than 1 N. During validation, the generalization potential of the proposed methodology with respect to previously unknown objects was demonstrated with an accuracy of 0.4–2.1 N and precision of 1.7–3.4 N, respectively.