Compositionality for hierarchical reinforcement learning

Recent breakthroughs in AI have proven that reinforcement learning can be used successfully to solve complex sequential decision problems to achieve near-optimal solutions. However, despite the successful applications that use function approximation techniques, RL algorithms face open challenges that involve data efficiency and generalization. In this line, hierarchical methods have been historically applied to simplify learning as much as possible. In this thesis we study how algorithms can exploit compositionality properties to obtain the optimal solution along with a hierarchical decomposition of a large problem. To do this, we devise algorithms in which the agent works at different levels of abstraction to learn a series of base behaviors. These can be combined to optimally solve a higher level task specification. Such base behavior can be reused in different ways to bring computational efficiency by reducing the number of learning samples used to solve each problem.