Comparative Study on Curiosity with Attention, Memory and Empowerment

Currently, most Deep Reinforcement Learning agents train their policy via the feedback signal provided by the environment, during the agent-environment interaction loop. This feedback, in the form of a reward signal, is usually task specific and manually hand engineered. However, in many real-world scenarios of interest, it is very hard or even unfeasible to manually craft this reward function. In other cases, the reward signal provided to the agent may be too sparse, hindering the learning process. Curiosity is a research field that aims to address these issues by endowing the agent with an intrinsic reward signal, allowing it to continue to explore its environment even in the absence of an external feedback signal. This work revisits the Intrinsic Curiosity Module, a formulation of curiosity based on the prediction error, and augments it with techniques from other research areas, such as attention and memory. An agent based on empowerment is also used in this comparative study, for completeness. The Atari 2600 videogame benchmark was used to perform and validate all the experiments.