Fig 1.
Graphical models for gaze following.
(a) The graphical model shows the probabilistic dependencies between different random variables in the model: G = goal, A = action, Xi = current state, and Xf = final state. The model captures how actions depend on goals and states, and how the state changes as a consequence of executing an action; (b) incorporates the influence of blindfold self-experience on the model using the random variable B; (c) shows the combined graphical models, one for the agent and a copy for the mentor (superscript m), for following the gaze of a mentor. Shaded variables denote observed variables. The darker shading indicates that B is an observed discrete variable, while the rest of the nodes are continuous.
Fig 2.
Robotic tabletop organization task setup.
(a) The robot is located on the left side of the work area and the Kinect looks down from the left side from the robot perspective. The three predefined areas that distinguish object states are notated. (b) Toy tabletop objects.
Fig 3.
Graphical models for robotic goal-based imitation.
(a) through (f) illustrate the use of graphical models for learning state-transitions, action inference, goal inference, goal-based imitation, and state prediction. Shaded nodes denote observed variables.
Fig 4.
The agent and mentor face each other in a 2D simulated environment. Goal positions inferred by the model from mentor observations are shown in red next to true goal positions (blue) for sampled goal positions both to the left and to the right of the agent. Black arrows represent the initial and final gaze vectors of the agent and mentor for one of these test data points. In this simple example, the goal locations were equidistant from the agent and mentor but the model readily generalizes to other cases as well.
Fig 5.
Accuracy of goal inference and gaze following.
Histogram plots showing the probability of an error (in degrees) between the inferred and the true gaze vector–the gaze vectors in the final state Xf were used for (a) and (c), and the gaze vectors in the the goal state G were used for (b). These probabilities were estimated from 375 test points spread uniformly over the test region. Note how the accuracy gracefully decays as more complicated inference is performed ((a) is the simplest, while (c) is the most complex).
Fig 6.
Comparison of the model to infant data from [6].
Error bars represent standard error of means.
Fig 7.
Robotic task: Learned transition model.
Each row represents different types of objects, and each column represents the different initial states Xi. The colors of bars represent different final states Xf. The y-axis represents the range of probabilities and the x-axis represents the six different manipulation actions available to the robot (PL = place LEFT, PR = place RIGHT, PO = place OFFTABLE, UL = pUsh LEFT, UR = pUsh RIGHT, UO = pUsh OFFTABLE). We do not show actions that cause self-transitions given an initial state.
Fig 8.
(a) Most likely goals: Initial and final states are at the top of each column. The height of the bar represents the posterior probability of each goal state, with the true goal state marked by an asterisk. (b) Inferring actions: For each initial and desired final state, the plots show the posterior probability of each of the six actions, with the MAP action indicated by an asterisk. (c) Predicting final state: The plots show the posterior probability of reaching the desired final state, given the initial state and the corresponding MAP action shown in (b). The red bar marks 0.5, the threshold below which the robot asks for human help in the Interactive Goal-Based mode.
Table 1.
Comparison of approaches.