Robotic Grasps of Cylindrical and Cubic Objects via Real-Time Learning-Based Shape Detection

Robots grasping objects are critical capabilities in warehouse environments and industrial settings. A robotic grasp generally occurs in a scenario where it is unfeasible for a worker to efficiently complete a tedious task, such as picking food and drink cans (cylinder-shaped) and packaging boxes (cube-shaped). It is worth noting that the tops of cylinders and cubes can be represented by ellipses and rectangles in the two-dimensional (2D) space, respectively. Therefore, a robot can grasp cylinder-shaped and cube-shaped objects by ellipse and rectangle detection. However, it faces the challenge of how to accurately detect cylindrical and cubic objects in real-time for robot grasping. To tackle the above research problem, we propose a grasping system that enables a robot to grasp cylinder-shaped and cube-shaped objects in static and dynamic environments by the proposed ellipse and rectangle detector. An end-to-end learning model is constructed to first incorporate a one-stage detection backbone and then, accommodate the proposed adaptive multi-branch multi-scale net with a designed iterative feature pyramid network, local inception net, and multi-receptive-field feature fusion net to generate object detection recommendations. Employing depth information, the coordinates of detected objects are converted to the 3D space via sampling a series of registered depths and pixels on objects from the live video stream. Comparisons with recent detection methods on the same dataset indicate that the proposed ellipse and rectangle detectors present better performance. Abundant grasping experiments are conducted to illustrate that a robot, empowered by the proposed detector, has the capability of grasping cylindrical and cubic objects in dynamic scenarios. (Video on YouTube, <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://youtu.be/KK1OtW6GvL0</uri>). Note to Practitioners—This paper is motivated by the problem of how to enable a robot to grasp objects with the basic geometric primitives-ellipses and rectangles in static and dynamic scenarios. Our target is to provide a potential solution for flexible industrial settings in operating moving cylinder-shaped and cube-shaped objects (food and drink cans and packaging boxes) in dynamic scenarios such as conveyors of production lines and logistics lines. We constructed a supervised learning model that can accurately and quickly detect ellipses and rectangles. Through the verification of the comparisons with recent methods and robotic grasping experiments, the behavior of the proposed method can be used in practical applications. In the future, we will deploy this robotic grasping system based on the proposed perception method to grasp food and drink cans and packaging boxes from moving conveyors on production lines and logistics lines.