Methodology on robot-based complex surface processing using 2D and 3D visual combination

Despite the significant development of automated equipment in the seafood industry, including the automated processing equipment for fish and crab, tremendous challenges accompany. Low processing capability for complex seafood surfaces is a paradigm, leading to heavy reliance on manual labor. The research is driven by the urgent need for robotic processing of the complex shape of seafood, which has a promising potential in the processing of objects with random, flexible, and complex shapes. In this paper, we are proposing a novel approach to processing complex surfaces designed for the seafood industry by combining 2D and 3D visual information based on point cloud segmentation. Porcupine crabs—a species of king crab in the family Lithodidae living in the Canadian Atlantic Ocean—have complex surfaces with long, sharp spines and are chosen as the case study. The unique feature of the Porcupine crabs poses substantial challenges to the conventional visual processing method in terms of low accuracy and efficiency. On the other hand, using our method, the crab feature has been successfully recognized and processed by cutting the spines. The robot spine removal tool path is generated based on the extracted spine features. Simulation results using Robot Operating System (ROS) and experimental tests have validated the robustness of the proposed method.