Cognitive Escape Reinforcement Learning for Complex Decision Making

Reinforcement learning (RL) has achieved unprecedented success on making complex decisions encountered in various applications. However, RL has two well-recognized technical challenges, namely, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">weak scene generalization</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">local-optima trapping</i>; both lead to degraded performance in terms of navigation accuracy and efficiency. To tackle these issues, a novel RL framework is proposed in this paper, called the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">cognitive escape reinforcement learning</i> (CERL), which is inspired by the cognition ability inherited in the human beings. Our developed CERL framework consists of two sequential stages: the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">semantic cognition module</i> (SCM) and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">cognitive escape module</i> (CEM), corresponding to the roles played by the perception layer and the decision layer in traditional RL, respectively. The SCM exploits <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">semantic</i> information extracted from the scenes for facilitating the CEM on making complex decisions. Our developed SCM is based on the semantic information extracted from the scenes that is quite capable of adapting to new environments for improving the agent's navigation performance. The CEM proactively identifies the occurrences of local-optima trappings and escapes from such incidences responsively. Extensive experiments have demonstrated the superiority of our proposed RL method, CERL, when compared to the state-of-the-art methods.