Action-Based Contrastive Unsupervised Representations for Reinforcement Learning Toward Robotic Manipulation

Although deep reinforcement learning (DRL) has made massive progress in policy learning, its reliance on a large number of real-world data samples presents a significant barrier to broader practical applications. To improve data efficiency, we propose an innovative algorithm, action-based contrastive unsupervised representation learning (Action-CURL). This method builds on the fundamental principles of the classical CURL algorithm by introducing a novel loss function that quantifies the correlation between continuous states while integrating action information into the model as a constraint. In this work, we tackle the challenge of efficiently learning robotic manipulation policies from visual observation data through a critical appraisal of the inherent limitations of DRL algorithms when applied in real-world scenarios. Our goal is to develop a more efficient and pragmatic method for robotic manipulation policy learning that can effectively mitigate the data scarcity challenge in DRL. Empirical evidence from our experiments indicates that the proposed method outperforms the original CURL algorithm across a diverse range of simulated and real-world environments. On average, our method achieves comparable performance to CURL in just 58.85% of the time across the DMControl suite’s 500k benchmark.