Exploring Visual Reinforcement Learning for Sample-Efficient Robotic Manipulation

Sample-efficient reinforcement learning (RL) from high-dimensional sensor readings holds significant potential for real physical robot applications. Recent developments in visual RL, which learns directly from image observations, have narrowed the gap between image-based and state-based training, especially in handling continuous actions. In this paper, we explore the potential of visual RL for achieving sample-efficient robotic manipulation. We evaluate four state-of-the-art visual RL algorithms-CURL, DrQ-v2, TACO, and DrM-on three robotic manipulation tasks from the DeepMind Control manipulation suite: ‘reach-duplo,’ ‘push-box,’ and ‘lift-box.’ Our findings reveal that while visual RL shows promising sample efficiency for robotic manipulation, it still lags behind state-based RL methods that learn from low-dimensional state vectors.