Causal Actions Recognition for Efficient Exploration in Robotic Reinforcement Learning

When using deep reinforcement learning to handle a robot task, it is necessary to learn a policy that outputs the actions the robot should execute. This policy needs to be learned through the robot’s continuous trial-and-error interactions with the environment. The robot may have the capability to perform multiple actions, but in fact, it doesn’t need to use all of these actions to complete the current task. It can reduce unnecessary exploration during reinforcement learning by using only those actions that impact task completion. This paper introduces a causality-based approach to identify essential robotic actions for task completion, utilizing environmental interactions for more accurate causal action identification than manual selection. And an experimental comparison on a multi-stage task demonstrates that constructing the action space using only the robot’s causal actions can facilitate more effective learning.