Enhancing Antipodal Grasping Modalities in Complex Environments Through Learning and Analytical Fusion

The robotic pick-and-place system is applied widely in many fields such as assembly, packaging, bin-picking, and sorting. In this paper, we present a deep learning and analytical-based method for generating antipodal grasping multi-modality in highly cluttered scenes. Our method takes advantage of three types of grasp poses to deal with the complexity of environment and achieves efficient computation time for real applications. A new synthetic training datasets are generated in Isaac Sim including approximately 35000 RGB-D images and an automatic labeling algorithm is developed. We utilize convolutional neural networks (CNNs) for predicting antipodal grasping parameters on objects and a filtering algorithm to avoid collisions and calculate grasp’s depth simultaneously. Our approach processes the entire task in approximately 0.2 seconds, achieving a success rate of over 96% and more than 98% collision-free grasps in cluttered scenes. The method was verified by experiments with RB10 robot arm, two-fingers grippers, depth camera L515, and several objects in different scenes. The article shows a simple, effective, and highly applicable approach in real environments. The real experimental video of our method is shown at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://www.youtube.com/watch?v=GvJZxUyQr3w</uri>. Note to Practitioners—The robotic pick-and-place task is fundamental to the progress of automation. This article presents a novel method for efficiently picking objects from complex environments characterized by obstacles, overlaps, and a diversity of objects. These scenarios pose challenges in approaching objects and avoiding collisions. Our method combines learning and analytical approaches, demonstrating high performance in terms of accuracy, generalization across various scenes and grippers, and rapid computation, with the entire process taking approximately 0.2 seconds. These advancements are validated through a series of real robotic grasping experiments and comparisons with state-of-the-art methods. We believe that our method represents a significant contribution to the field of automation science and holds promise for real-world applications.