Q-Vector: A General 6-DoF Grasping Framework Based on Self-Supervised Deep Reinforcement Learning

Self-supervised learning is a promising method for grasping unseen objects. However, most existing methods are constrained in the top-down planar grasps due to the dimension explosion of state and action space. In this paper, we propose a multi-Q learning framework, called Q-vector, for 6-DoF grasping of unseen objects in clutter. Through decoupling the action space into different parts, the grasping problem is formulated as separated Q-value policies which could be learned with easy reward engineering and fast convergence speed. To keep the inherent relationship among different parts of action space during policy learning, we design a serial-parallel-mixed neural network model. Further, we propose a curiosity exploration mechanism based on curvature ratio of scene point clouds to improve the sample efficiency. Experimental results show that Q-vector achieves the satisfied grasping performance for unseen objects compared with its peers and demonstrates the generalization capability when transferred to the real setup without any additional training.