Deep Active Inference with Generative Actions and Diversity-Based Action Choice

The literature of Deep Active Inference, implementing the generative, biologically inspired Active Inference framework with the Deep Learning approach, often makes use of a hidden state transition model to generate current hidden states. It also usually leverages the Monte Carlo family methods to choose the agent's next action that minimizes the Expected Free Energy. The action identification typically uses either a stochastic sampling process or a learning of sampled actions by a ‘habit’ model. In this work, the goal is to explore an approach based on the learning and generation of actions as a result of hidden state transitions. The corresponding generative model, along with the variational form of the Free Energy and the Expected Free Energy, are formulated for an environment represented as a Partially Observable Markov Decision Process, and the model architecture is also presented. We also suggest a novel approach for the action choice: the generated action minimizing the Expected Free Energy is chosen based on the diversity of the expected risk relatively to that of its originating action set. The Active Inference agent is also equipped with top-down, selective, context-dependent attention mechanisms to control its behavior. Experiments have been conducted by addressing the continuous versions of Mountain Car and Inverted Pendulum problems. The results show the ability of the agent to learn and solve both problems with promising performance, requiring noticeable changes only on high-level attention parameters. This work highlights that this approach of action generation, choice and planning by Active Inference agents might represent a worthy alternative to usual methods, noticeably for considerations of computational efficiency and bio-mimetism.