Real-Time Object Grasping and Placement in Dynamic Environments via Model-Based Policy Gradient

This paper proposes an advantage policy with imitation (API), a model-based reinforcement learning method for realtime object grasping and placement in dynamic scenarios. The method employs a policy gradient update guided by a novel multisampling group relative advantage estimation (MGRAE) mechanism. MGRAE replaces conventional generalized advantage estimation by computing advantages through action-group sampling. This approach improves computational efficiency and eliminates the dependency on value networks during policy updates. Additionally, a behavior cloning subtask leverages planned actions to supervise policy training, helping mitigate policy lag. The pushcube and pickcube experiments in the ManiSkill3 robotic manipulator environment demonstrate that API outperforms baseline methods (TD-MPC2 and PPO) in training efficiency and task performance.