Metaverse-Ready Simulation: Advancing Sim-to-Real by Reinforcement Learning for TurtleBot in ROS2 and Unreal Engine

Metaverse platforms now offer robotics researchers sophisticated avenues to construct photorealistic, interactive training contexts that transcend constraints of traditional simulators. We present an integrated reinforcement learning framework coupling ROS2 with Unreal Engine for training TurtleBot3 agents under diverse, complex operational conditions. These configurations emphasise advanced visual fidelity and physics-based modelling, critical for assessing real-world performance in controlled yet rich virtual environments. Our empirical comparisons with Gazebo, employing the Deep Deterministic Policy Gradient (DDPG) and Twin Delayed Deep Deterministic Policy Gradient (TD3) over 2,000 episodes, indicate that learning outcomes are effectively identical across both simulators. This confirms Unreal Engine as a robust alternative with negligible compromise in performance metrics. Additionally, it furnishes superior rendering, real-time ROS2 interoperability, a direct link to robotics middleware, and straightforward asset provisioning through Fab—benefits particularly advantageous for high-level Sim-to-Real transfer, where graphical and physical fidelity narrow the gap between simulation and tangible deployment. By aligning with metaverse frameworks necessitating continuous, shared, and digitally enriched environments, our system supports more adaptive and scalable robotics research. In sum, uniting ROS2 with Unreal Engine, complemented by elevated photorealism and effortless asset access, signifies a major progression towards refined simulation schemes suited to emerging metaverse-centred robotics applications.