A Cross-Modal Approach to Shape Reconstruction of Deformable Linear Objects

This research focuses on applying cross-modal perception to reconstruct the shape of cables, which are treated as DLO (Deformable Linear Objects). The adoption of multiple sensing modalities --visual and tactile in this work-- is crucial in those situations where the cable is only partially visible due to occlusions or camera faults. The challenge is twofold: finding a common representation for the two sensing data to obtain a single DLO model, e.g., a spline, and allowing the robot to explore the cable by autonomously deciding the sensing modality to adopt. The method is experimentally tested on a robot manipulator equipped with an RGB-D camera and a tactile sensor to reconstruct the cable shape from simple straight lines to more complex shapes, e.g., spiral-like shapes.