A Study on Bridging the Gap With Reinforcement Learning: TD3 Optimization for Sim-to-real Transfer from Gazebo to Isaac Sim

This study addresses the critical challenge of Sim-to-Real transfer in reinforcement learning by analyzing the performance gap between two widely used simulators, Gazebo and Isaac Sim. We first demonstrate that a standard TD3 (Twin Delayed Deep Deterministic Policy Gradient) agent, while effective for a mobile robot navigation task in the physics-focused Gazebo environment, fails to converge in the more visually and physically complex Isaac Sim environment. To bridge this performance gap, we propose a two-fold optimization strategy for Isaac Sim: first, a fine-tuned reward function to provide denser learning signals, and second an enhanced state representation pipeline that uses voxelization and a convolutional block attention module (CBAM) to efficiently extract salient features from high-dimensional 3D LiDAR data. Experimental results validate our approach, demonstrating that fine-tuning the reward function improves the mission success rate by 24%, while the subsequent integration of the CBAM-based feature extractor contributes an additional 13% improvement. This research empirically proves that environment-specific optimizations are crucial for high-fidelity simulators and provides a structured methodology for adapting RL agents to these complex settings, offering valuable insights for robust Sim-to-Real transfer.