Deformable object manipulation based on double Q-map PPO

The manipulation of deformable objects for a manipulator poses significant challenges due to their high levels of uncertainty and complexity, making it difficult for traditional robotic arm control methods to achieve accurate and stable grasping. To address this issue, this paper proposes a Double Q-Map Proximal Policy Optimization (DQMPPO) method for manipulation of deformable objects. In DQMPPO, the manipulation policy is divided into two parts: a position evaluation policy and a pose grasping policy. The former is responsible for assessing the value of potential grasping locations on the deformable object to determine the optimal grasping point. The latter, on the other hand, generates appropriate robotic arm movements to achieve stable grasping and manipulation of the deformable object. The training methodology of DQMPPO is similar to that of the classical Proximal Policy Optimization (PPO). This paper builds a simulation environment based on Softgym and validates the proposed method. The experimental results show that compared with existing methods, the proposed method in this paper has higher accuracy and robustness.