Three-Dimensional Shape Estimation of a Soft Finger Considering Contact States

To achieve precise in-hand manipulation and feedback control using soft robotic fingers, it is essential to accurately measure their deformable structures. In particular, estimating the three-dimensional shape of a soft finger under contact conditions is a critical challenge, as the deformation state directly affects manipulation reliability. However, nonlinear deformations and occlusions arising from interactions with external objects make the estimation difficult. To address these issues, we propose a soft finger structure that integrates small magnets and magnetic sensors inside the body, enabling the acquisition of rich deformation information in both contact and non-contact states. The design provides a 15-dimensional time-series signal composed of motor angles, motor currents, and magnetic sensor outputs as inputs for shape estimation. Built on the sensing signals, we propose a mode-selection-based learning approach that outputs multiple candidate shapes and selects the correct one. The proposed network predicts the three-dimensional positions of four external markers attached to the finger, which serve as a proxy representation of the finger’s shape. The network is trained in a supervised manner using ground-truth marker positions measured by a motion capture system. The experimental results under both contact and non-contact conditions demonstrate that the proposed method achieves an average estimation error of approximately 4 mm, outperforming conventional one-shot regression models that output coordinates directly. The integration of magnetic sensing is demonstrated to be able to enable accurate recognition of contact states and significantly improve stability in shape estimation.