V-HCP: A Robust Visual-Haptic Compensatory Perception Network for Flexible Printed Circuits Assembly

In the assembly process of flexible printed circuits (FPCs), the successful docking of the FPC connector and receiver by robot arms relies on accurate estimation of receiver contours. However, the mainstream visual positioning methods are mainly aimed at large, well-characterized, and rigid objects, and it is a challenge to apply to the assembly of tiny and flexible components such as FPCs. Additionally, the haptic information is of low quality or even absent in multimodal methods. Motivated by these problems, we propose the robust visual-haptic compensatory perception (V-HCP) Network to fuse stable visual and high-precision haptic information. Specifically, we introduce a compensation module to address modality absence. First, to address the imbalance in contributions from the two modalities due to differing dimensional characteristics, we employ Lanczos resampling to upsample the haptic data. Considering the limited data availability resulting from the high cost of experimental data acquisition, we utilize a Siamese network structure to compare ideal and real input features. Second, to compensate for the potential loss of haptic information, we introduce a robust compensation branch (RCB), leveraging shared features from the visual modality to supplement the missing haptic data. Experiments conducted on the FPC assembly platform of the Redmi Note 11 smartphone demonstrate that our proposed V-HCP achieves a remarkable success rate of 98.5%, surpassing the performance of existing single-modal models. Furthermore, the model maintains accurate predictions even in scenarios where haptic information is unavailable, indicating the improvement of the robustness.