Shape Estimation of Viscoelastic Fluid Materials: An Application for Weld Pool

Understanding the behaviour of manipulated objects is an essential step in robotic-vision applications, particularly if these objects are of a deformable nature. Many of the research carried out in the field of shape and state estimation of deformable objects focused on investigating consistent materials such as clothes and ropes. Unfortunately, the study of fluid-kind materials as deformable object, is not being as much studied. In this paper, we focus on fluid deformable objects shape and state estimation. We introduce the use of a generative model based on a neural network to generate a relevance space that focuses on the most important regions in the image. We also introduce an algorithm for the semi-automatic generation of this relevance space. Concretely, the network architecture used here is based on a Variational Auto-Encoder (VAE), where a ResNet network is used as backbone. The ResNet network encodes an input image into a probabilistic latent vector of size 256 and a convolutional network decodes this vector to reconstruct a heatmap of the relevance space that represents the shape of the object. To evaluate our contribution, we employ cross-fold validation to study the feasibility of the proposed method. We apply this method for weld pool shape estimation. As a result of cross-fold validation, we conclude that our method shows good stability and good prediction of the shape of the weld pool, with an average error of 14.32px. Code is available on https://github.com/Oassal/robio2024