Fig 1.
(A-C) The three simulations included in the study. In all cases, the agent is seated with the forearm on the table. (A) In the first simulation, the agent has direct visual access to the real hand or equivalently, to a virtual hand projected to spatially overlap the actual hand position. (B) In the second simulation, the agent embodies a static fake hand placed in a configuration different from the real hand, hidden from view. (C) In the third simulation, the agent embodies a virtual arm seen through a stereoscopic visor and has to reach a target; the simulation includes a manipulation of the visuomotor mapping that, as the agent moves, introduces a spatial misalignment between the two hands. (D) In all the simulations, the agent is in control of one degree of freedom of the elbow, which can rotate along the vertical axis. The agent state is then uniquely described by the elbow joint angle θE, with the corresponding visual representation of the end-effector specified in Cartesian coordinates (xp, yp).
Fig 2.
Mathematical description of the generative process.
The system state vector (Eq E.1) describes the system in generalized coordinates at the first order. The sensory state vector (Eq E.2) maps the system states into the sensory input (here proprioception and vision). The forward mapping (Eq E.3) describes how the system state vector maps into sensory input. The system dynamics (Eq E.4) is expressed as a set of differential equations that describe the expected temporal unfolding of the system state. The arm dynamics is approximated as a damped system driven by a combination of external forces (FE) and agent’s actions (A); in Eq E.4 marm represents the forearm mass and ϕ the viscosity constant of the damping. We assume a power dependence between damping and velocity, with β = 0.5, which allows reaching plausible velocity profiles.
Fig 3.
Mathematical description of generative model, i.e. the internal model the agent holds of the system and its dynamics, for the 1D model shown in Fig 1.
The agent represents its own state via the internal state vector that describes the system in generalized coordinates at the second order (Eq M.1). The forward model, gμ(μθ), describes how the agent forms an estimate of the expected sensory input based on the inferred system state, μs (Eq M.2). The model of the system dynamics allows the agent to predict the temporal evolution of its own state. In particular, the agent in our model entails a representation of reaching actions as instantiations of a desired state, , which acts as an attractor. The dynamics is then assumed to follow that of a damped oscillator (Eq M.3), with K representing the elastic constant controlling the attraction strength, ϕ the viscosity constant of the damping, and marm the mass of the agent forearm. Note that the model in Eq M.3 describes also states in which the agent does not intend to move; in this case the desired state is to be set to the current state, i.e.
.
Fig 4.
Perception-action loop of the active inference agent.
The variables and equations in black (blue) are from the generative process (generative model), thus represent the real world (internal representations the agent holds about the system state and its dynamics). See the main text for a detailed explanation.
Fig 5.
Derivation, under the Laplace approximation, of the variational free energy (VFE), , for the active inference agent.
See S1 Appendix for the detailed derivation of Eq F.1, in which is expressed in terms of the Laplace encoded energy, and C is a term assumed here to be a constant that encodes the optimal variance (omitted for clarity as it will not be used for computing the gradients as it does not depend on the internal state μ nor the action A). p(sp|μ) and p(sv|μ) are the proprioception and visual likelihood given the internal belief μθ.
is the joint probability of the internal state vector up to 2nd order, which can be expressed in terms of conditional probabilities on the system state as expected from the internal dynamical model. Both the sensory state likelihood and the conditional probabilities are approximated as Gaussians centred on the expected sensory state and the expected value of the dynamics at the different orders (from Eq M.3 in Fig 3) respectively. Please see the main text for a more detailed explanation.
Fig 6.
Derivation of the free energy gradients for the active inference agent.
Please see the main text for a detailed explanation.
Fig 7.
First simulation: reaching a fixed target in space.
(A) Schematic representation of the agent having direct vision of his own arm, which initially rests on a table surface and could only move by rotating the elbow around the vertical axis. (B) Task specification. The agent has to reach a target location (green star) by rotating the arm so to reach the configuration shown in grey, at θT, corresponding to having the hand at the target location [xT, yT] = g(θT). (C-G) Dynamics of the model variables during the task. The vertical bars mark the time at which the target location is disclosed. (C) Joint angle of the real (black) and inferred (red) arm configurations, expressed in radians; the green line represents the arm configuration θT for which the hand is on the target. (D) Real (black) and inferred (red) velocity of the elbow joint angle velocity. (E) Action, represented in our model as an angular acceleration. (F) The three prediction errors considered in the model: model dynamics error (green) and the two sensory errors, proprioceptive (magenta) and visual (blue). Please note that in this plot, the prediction error units correspond to different dimensions and cannot be directly compared. (G) Contributions of proprioceptive (magenta) and visual (blue) errors in determining the action, more specifically the two vector components in the r.h.s. of Eq G4 in Fig 6. See the main text for explanation.
Fig 8.
Second simulation: multisensory conflict in the rubber hand illusion (RHI).
Schematic representation of the agent seeing a virtual hand through a virtual reality stereoscopic headset. (B) Task specification. The agent undergoes a body ownership illusion over a virtual arm shown with a rotation θT with respect the real arm initial position. (C-G) Dynamics of the model variables during the task. The vertical bars mark the time of the illusion onset. (C) Joint angle of the real (black) and inferred (red) arm configurations, expressed in radians; the blue line represents the virtual arm configuration θT. (D) Real (black) and inferred (red) velocity of the elbow joint angle velocity. (E) Action, represented in our model as an angular acceleration. (F) The three prediction errors considered in the model: the model dynamics error (green) and the two sensory errors, proprioceptive (magenta) and visual (blue). Please note that in this plot, the prediction error units correspond to different dimensions. (G) Contributions of proprioceptive (magenta) and visual (blue) prediction errors in determining the action, more specifically the two addends in the r.h.s. of Eq G4 in Fig 6. The gray line represents the potential visual contribution that is set off by assuming that kA,v = 0. See the main text for explanation.
Fig 9.
Third simulation: reaching under visuo-proprioceptive conflict that cannot be resolved by acting.
(A) Schematic representation of the agent seeing a virtual hand through a virtual reality stereoscopic headset. (B) Task specification. The agent has to reach a target placed at [xT, yT] = g(θT), while undergoing an illusory ownership over a virtual hand, which moves along the real hand but with a velocity gain of 1.3. (C-G) Dynamics of the model variables during the task. The vertical bars mark the time at which the target location is disclosed. (C) Joint angle of the real (solid black) and inferred (solid red) arm configurations, expressed in radians. The green line represents the desired arm configuration θT for which the hand is on the target. (D) Real (black) and inferred (red) velocity of the elbow joint angle. (E) Action, represented in our model as an angular acceleration. (F) The three prediction errors considered in the model: model dynamics error (green) and the two sensory errors, proprioceptive (magenta) and visual (blue). Please note that in this plot, the prediction error units correspond to different dimensions. (G) Contributions of proprioceptive (magenta) and visual (blue) errors in determining the action, more specifically the two addends in the r.h.s. of Eq G4 in Fig 6. (H) Real (solid curves) and inferred (dashed curves) velocities of the elbow joint angle for different simulations that use different values for the velocity gain, which maps the real elbow joint angle velocity into the corresponding join angle velocity of the virtual arm. The black line, corresponding to a velocity gain 1, is the same as in the first simulation. The inset permits appreciating the differences between the inferred velocities. See the main text for a more detailed explanation.