Learning action-oriented models through active inference
Fig 2
(A) Agent overview. Agents act in an environment which is described by states ψ, which are unknown to the agent but generate observations o. The agent maintains beliefs about the state of the environment s, however, s and ψ need not be homologous. Agents also maintain beliefs about control states u, which in turn prescribe actions a. Finally, the agent maintains beliefs over model parameters θ, which describe the probability of transitions in s under different control states u. (B) Actions. At each time step, agents can either run, which moves them forward one unit in the direction of their current orientation, or tumble, which causes a new orientation to be sampled at random. (C) Approximate posterior. The factorization of the approximate posterior, and the definition of each factor. In this figure, x denotes the variables that an agent infers and ϕ denotes the parameters of the approximate posterior. We refer readers to Methods section for a full description of these distributions. (D) Generative model. The factorization of the generative model and the definition of each factor. Here, λ denotes the parameters of likelihood distribution and α denotes the parameters of the prior distribution over parameters. We again refer readers to the methods section for full descriptions of these distributions. (E) Free energy minimization. The general scheme for free energy minimization under the mean-field assumption. We refer readers to the Methods section for further details. (F) Control state inference. The update equation for control state inference, where . This equation highlights the difference between the three action-strategies considered in the following simulations.