Deep Reinforcement Learning for Sim-to-Real Transfer in a Humanoid Robot Barista

In recent years, an increasing amount of research and development efforts have been invested in home-service humanoid robots. Nevertheless, the complexity of home environ-ments poses challenges in the robustness of robot manipulation with high accuracy and swift promptness. In this paper, we study the coffee-making application as a demonstrative example to enable humanoid robots in the edge-cloud cooperative intelligence architecture. We propose a reinforcement learning robot manipulation method with visual perception for filling-up the sim-to-real gap. We construct a high-fidelity coffee making digital twin simulation environment. In addition, we extract the key points of robot hands using computer vision algorithms to achieve consistency between the simulation and the real scenes. The domain randomization has been applied to enhance the policy tolerant to the positional deviation of the coffee machine and coffee cups. We deploy the trained policy to the real robot in a zero-shot transfer. Our experimental results have demonstrated that the proposed method achieved over 90% success rate in end-to-end coffee making tasks on both simulation and real robots. The proposed method significantly outperforms other image-based or point cloud-based methods during the sim-to-real process.