RoPESim: A Framework for Robot Manipulation Policy Evaluation via Simulation

Predicting the robot manipulation plan prior to real-world execution is an important capability for robots to complete tasks in manufacturing environments. However, current AI-based manipulation planning methods lack this capability, making it difficult to deploy them in real-world manufacturing scenarios. In this work, we propose a simulation-based human-robot collaboration framework to evaluate predicted robot actions before real-world execution. The framework consists of a VLM-based scenario generator, a diffusion-based action simulator, and an evaluator. First, the scenario generator automatically creates a simulation scenario with objects and obstacles identified and placed. Then, the action simulator generates a series of manipulation action trajectories using a diffusion model in the simulation environment. Each action trajectory is assessed by the evaluator for collision failure, manipulation failure, and completion rate. The final evaluation results are returned to the user for verification and approval. In our experiment, we apply our framework to five chosen scenarios with highly potential collision failures. For each scenario, at least one feasible planned action trajectory is generated. It is then verified through real robot execution, demonstrating the effectiveness of the proposed framework.