An End-to-End Multi-Dimensional Perception Network Architecture for Robotic Grasp Detection with Target Edge Collision-Aware Strategy

This paper investigates the feasibility of robotic grasping of various objects in complex scenarios, with our method aiming to achieve grasping capabilities for any scene and any object. Firstly, a Target Edge Collision-Avoidance Strategy is proposed that systematically incorporates the edge features of grasping objects. This strategy is specifically designed to address two critical challenges: bridging the significant performance gap between offline training data and real-world operating conditions, and effectively preventing collision incidents between the robotic end-effector and target objects during physical grasping operations. Furthermore, the Grasp Detection Network based on Global and Local Information Perception (GLIP-Net) is proposed, featuring two intricately designed components: the Global Information Perception and Local Aggregation Module, and the Multi-dimensional Multi-scale Attention and Adaptive Feature Fusion Module. The GLIP-Net enhances the network’s perceptiveness to global information, strengthening the correlation between features and the spatial parameters of grasping. To validate the effectiveness of the presented method, extensive tests and grasping experiments is conducted on the Cornell Dataset and Jacquard Dataset, as well as in practical scenarios. The empirical outcomes indicate an accuracy level of 99.2% on the Cornell Dataset and 96.8% on the Jacquard Dataset, respectively. Furthermore, by employing the Kinova robot in both single-object and multi-object complex scenarios within real-world environments, the grasping success rates of 97.0% and 95.8% is achieved. Note to Practitioners—This paper was inspired by the problem of robotic object grasping in various scenarios, but it is also applicable to tasks such as object grasping, sorting, and transportation in unstructured environments. Existing robotic grasping methods are typically limited to structured scenarios, where robots can only grasp objects at fixed positions. When the object or its pose changes, the entire grasping task is likely to fail. Additionally, robots often fail to consider the edge information of the object when grasping, leading to collisions between the end effector and the object. In this paper, we propose a novel approach that employs a graping detection network to process an input color image containing depth information. The neural network takes both global and local information into account, fuses useful feature data, and adaptively outputs a set of grasp configurations. To address the issue of collisions between the end effector and the object, we design a target edge collision-avoidance strategy, prioritizing regions adjacent to the gripper side. Preliminary experiments demonstrate the feasibility of our method, which has been tested in several real-life scenarios. In future research, we aim to address the robot vision closed-loop control problem, enabling robots to perform grasping tasks in dynamic environments.