Subjects, task, and data

Task.

Two monkeys sat in primate chairs (Crist Instruments, Damascus, MD) modified to allow access to joysticks (IP66s, CTI Electronics, Stratford, CT) affixed in a custom-built acrylic case at the front. All axes of the joystick were calibrated with a MATLAB (Natick, MA, USA) script to ensure that every direction yielded equal speed. Horizontal and vertical eye positions of the shooter were sampled at 1000 Hz by an infrared camera (Eyelink 1000, SR Research, Kanata, Ontario, Canada). The task was presented and data were collected via custom-built scripts in MATLAB using the Psychophysics and Eyelink Toolbox extensions [20] (see http://psychtoolbox.org/). Primate chairs were oriented at a ninety-degree angle to one another such that each monkey could see the other to his left or right. Each monkey also faced his own computer monitor displaying the task; the display was the same on both screens.

Each trial was preceded by a 300ms period during which the monkeys each had to center their joysticks. Then a green circle, the puck, appeared at the left side of the screen, centered vertically. A long red rectangle, the goalie bar, appeared at the vertical center near the right of the screen. Further to the right, a gray line, the goal, extended from the top to the bottom of the screen. The screen was otherwise black (Fig 1). The “shooter” monkey’s joystick controlled the movement of the puck, which was unconstrained in all directions except that it could not go beyond the borders of the screen, and the “goalie” monkey’s joystick controlled the movement of the bar, which was similarly unconstrained vertically but did not move horizontally. The monkeys’ roles were fixed throughout an experimental session. Each trial ended when the puck reached the goal, when the puck hit the goalie bar, or when ten seconds had elapsed with no movement from the shooter. The horizontal and vertical location of the puck, as well as the vertical location of the goalie bar, were recorded throughout the trial at roughly 60 Hz. If the puck reached the goal, the shooter was considered to have won and he received a drop of juice in his mouth. If the puck hit the goalie bar, the goalie was considered to have won and he received a drop of juice in his mouth. Juice volume on each successful trial was similar for both monkeys and constant throughout the session. Following juice delivery, the screen went blank for a 1.5 s inter-trial interval before the next trial began. The speeds of the puck and goalie bar were titrated by the experimenter between gaming blocks (roughly 200 trials) to achieve a roughly 50% win rate for each player in each session. Monkeys were first trained against a computer opponent performing simple movements and subsequently confronted monkey opponents.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Penalty shot task and game play.

A: The two subjects viewed the same image on separate screens. One subject (at left here) played as the “shooter” and controlled the puck, while the other (at bottom here) played as the “goalie” and controlled the bar. B: Illustration of onscreen play for the two-player game. The puck (blue) is free to move in two dimensions, while the goalie (red) moves only up or down. Axes are x and y screen directions. Solid lines indicate player trajectories for a single trial. The goalie’s trajectory has been stretched along the x axis to indicate its movement through time. For animations of game play, see S1, S2, S3 and S4 Videos.

https://doi.org/10.1371/journal.pcbi.1006895.g001

Game dynamics

Let y_t be the observed variables of the onscreen trajectory at time t. Similarly, let s_t be the collection of system variables. We typically consider the latter to be positions and velocities (i.e., ), but they could additionally include accelerations, additional time points of lagged data, or variables encoding player identity or game condition. For the penalty shot task, y_t evolves according to (2) with v_max a vector of maximum velocities in each dimension as determined from the data, υ ∈ [−1, 1] a joystick position, and ⊙ the Hadamard (elementwise) product. We further assume that the joystick position υ is related to a latent control signal u via υ = tanh(u). That is, the observed control, restricted by the finite joystick range, may be less than desired control.

Control model

We assume that at each time t, u_t is the output of a control model attempting to minimize an error e_t ≡ g_t−y_t with g_t an instantaneous, state-dependent, set point for the controller that we will call a “goal”. These goals represent the instantaneous onscreen locations desired by each player, as evidenced by the fact that when y_t = g_t(s_t) (next goal and position are equal), we have e_t = 0 and thus no need for control. When e_t ≠ 0, however, we assume that changes in control are given by a proportional-integral-derivative (PID) controller: (3) with κ the proportional control constant and T_i and T_d the integration and differentiation time constants, respectively. That is, L is a convolutional filter, determined by κ, T_i, and T_d, learned separately for each player in each dimension. Changes in control are thus convolutions over the control error. Finally, to capture our uncertainty about this relationship, we model errors in control as normally distributed with variance ϵ²: (4) where the equation should be read as implying a separate, uncorrelated, normal distribution for each coordinate in u (with potentially distinct ϵ_i in each dimension). In practice, our onscreen observations have minimal noise, implying ϵ ≪ 1.

Unfortunately, Eq 4 is ambiguous, since for any sequence of controls u_t and any L, the equations for g are linear and thus in general invertible. And while this symmetry is weakly broken by our model for g (described below), which prefers some sequences of goals to others, we remove the ambiguity in practice by fixing control to be purely proportional early in training, consistent with the idea of a goal as a point toward which each player desires to move.

In addition, a second issue arises from the fact that joystick positions are constrained to lie in [−1, 1] along each dimension, while model-predicted control might be large. In our model, we achieve this by using a hyperbolic tangent function to link u (desired control) to υ (joystick input). However, this means that small changes in υ can result in potentially large changes in inferred u and thus g. More concretely, when joystick input is near maximal (υ ≈ ±1), this will be consistent for any goal farther than a certain distance from the player’s current location. Ideally, one would consider υ a censored version of u, a possibility we leave to future work. Here, we remedy this by imposing a penalty during model training on any goals that far exceed the visible game area.

Goal model

For the goal time series, g_t, we will assume a Markov process in which new goals are probabilistically selected at each time based on both the current goal and the current state of the system. That is, (5)

More specifically, we will assume that at each time point, there exists a function V(g_t, s_t) that captures the benefit in setting a particular goal based on the current state of the system. That is, we want to increase V as often as possible. (Alternately, we could consider −V as an energy function that we wish to minimize.) However, we also assume that the goal time series is relatively smooth, which we formalize by adding a regularization term for the rate of change between successive time points. More explicitly, let (6)

Here, u_*t ≡ u_t−1 + L*(g_t−y_t) is the the predicted control and ϵ and σ govern the control noise and goal diffusion, respectively. In what follows, we will also find it useful to define β ≡ σ⁻² and U(g) ≡ − σ² V in order to write log p(g) = −βE(g|s) − log Z′ with (7) which results from marginalizing the u_t out of Eq 6.

This formulation admits multiple interpretations: mostly simply, E(g|s) is the negative log probability of the goal time series: configurations that minimize this “energy” have higher probability. Along the same lines, the first term in Eq 7 is a penalty on large changes in g between time points, encouraging smoothness. In form, it is equivalent to a “kinetic energy” . Likewise, the second term, U(g), is a generator of changes in goal state. At each time point, goals are drawn toward regions of small U, making it analogous to a “potential energy.” Together, the probabilistic model over goal trajectories in Eq 7 is equivalent to a path integral for a particle with position g_t and energy K + U. In the limit of small V/small σ/large β, one is in either the low-temperature thermodynamic limit or the high-mass classical limit, and g is a spatially-varying perturbation of a Gaussian process. Alternately, in the limit of large σ, goals are simply chosen independently at each time point. In any case, we have made the strong assumption that dependence of g_t on g_t−1 occurs only through a momentum term, requiring that the “static” V term carries most of the weight of explanation.

Unfortunately, for general V(g|s), the distribution implied by Eq 7 is of the Boltzmann-Gibbs form and impossible to sample efficiently. If the goal of our inference is to model V itself, we will need a method for sampling from p(g) that still allows this partitioning.

Initial conditions

Let t = 1 be the time of the first observed data, y₁. In order to calculate y₂ using Eq 2, we need u₁, which from Eq 3 requires u₀, e₁, e₀, and e₋₁. (Recall that it is the joystick position υ₁ that is observed; u₁ is latent).

We bootstrap this process by assuming the following:

1. y_t≤0 = y_init, the vector of initial positions of both players.
2. For t < 0, g_t<0 = y₀, so there was no control error (e_t<0 = 0) and thus no control (u_t<0 = 0).
3. g₀ ∼ GMM(K₀). That is, the initial goal is drawn from a Gaussian mixture model with K₀ components. In our case, these component Gaussians, like those defining Eq 8, have diagonal covariance.
4. From point 2 above, e₋₂ = e₋₁ = 0. Using e₀ = g₀−y₀ then gives u₀ via Eq 3.
5. Using Eq 10 and g₀, we can then draw g₁ and calculate e₁ = g₁ − y₁. From Eq 3, it is then possible to calculate u₁ and thus y₂.

All future data points can then be calculated via the steps outlined above.

Inference

Given the observed system trajectory y_t, we would like to infer the underlying goal trajectory g_t. In general, full Bayesian inference is intractable, but we employ a variational Bayes (VB) approach [19, 21] that approximates this procedure. In brief, VB attempts to minimize the Kullback-Leibler divergence (a measure of difference between distributions) between a known generative model for which inference is intractable, , and an approximating family of posterior distributions, q(z). This is equivalent to maximizing an evidence lower bound (ELBO) given by (11) with the entropy of the approximating posterior. That is, inference is transformed into an optimization problem in the parameters of the approximate posterior q(z), amenable to solution by gradient ascent. In our model, we make use of so-called “black box” methods [22–25] in which the gradients of the ELBO are replaced with stochastic approximations derived by sampling from q(z), avoiding often difficult computations of the expectation in Eq 11. Thus our only requirement for the recognition model is that we be able to sample z_* ∼ q(z) and to compute log q(z_*).

In our case, we begin with the generative model specified by Eq 6. For the approximate posterior q(g|y), we use the variational latent dynamical system (VLDS) model of [26, 27]. (We are not interested in the posterior over u and implicitly marginalize over it). The VLDS is a nonlinear generalization of the linear state space model, implying that inference in the model is a generalization of the Kalman filter. It uses neural networks to flexibly parameterize the mean and covariance of the underlying time series, which are assumed to change dynamically. As in [24, 25], samples from this posterior are then used to update both the parameters of the generative model (L, ϵ, σ, μ, λ, w) and the parameters of the approximate posterior q via gradient ascent.

Change point analysis

We used the number of change points in the puck trajectory as a rough measure of the each player’s strategic complexity. Specifically, a time point t is a change point for a player if sign(υ_t) ≠ sign(υ_t+1), υ_t, υ_t+1 ≠ 0, and |υ_t − υ_t+1| ≥ 10⁻⁶. That is, joystick input changes sign, is nonzero both before and after the change point, and the difference exceeds a minimal value. We studied the correlation between number of change points and game results at both the trial and session level. To avoid over-identification of change points due to frequent variation in control signals, all trials were smoothed by the same Gaussian filter described in Data section before this analysis. We also excluded the first and last five time points to minimize edge effects.

Statistical testing

For testing autocorrelation functions, we applied the Ljung-Box test to the autocorrelation function calculated on each session. For initial velocities, we discarded periods of non-movement at the start of the trial, selecting instead the velocity at the first time point where the norm of the velocity exceeded 0.001. For analyzing whether outcomes of consecutive trials were correlated, we used Fisher’s Exact Test on game results (summarized by a 2 × 2 contingency table of win/loss of t^th and (t + 1)^th trials) by session. We corrected for multiple comparisons via Bonferroni correction to ensure a family-wise false positive rate of 1%.

Player tendencies

While players’ behavior was highly variable across trials and sessions (Fig 2A), blockwise adjustments to players’ maximum velocity kept win rates similar across players (Fig 2B). Likewise, the likelihood of a win was independent of whether the previous trial was a win or a loss (N = 4/120 sessions significant for dependence; Fisher’s Exact Test for independence; α = 0.01), suggesting little in the way of a “win-stay, lose-shift” or “hot hand” effect (Fig 2C). Moreover, while players’ initial velocities tended to cluster, their final positions were more evently distributed (Fig 2D and 2E). That is, players appeared to mix their play effectively. We found that initial velocities and final positions were only weakly autocorrelated (v_0x,shooter = 4/120, v_0y,shooter = 17/120, y_f,goalie = 21/120, y_f,shooter = 31/120 sessions significant; Ljung-Box test for autocorrelation at lag 1; α = 0.01 controlled for family-wise error rate) (Fig 2F), indicating that there was little strategic carryover across trials. In summary, while significant complexity and variation in play is apparent, this variation was not well captured by typical metrics derived from games with discrete action spaces.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Variability in player behavior.

A: Example trajectories from a single shooter, illustrating the diversity of player movements. B: Percentage of trials won by shooter for each behavioral session. Colors indicate shooter identity. Task parameters (puck speed, goalie size) were altered on a per-session basis to balance game play between players. C: Correlations in game outcome. Scatter plot of the probability of a shooter win as a function of shooter win or loss on the previous trial. Each dot represents one behavioral session. Colors indicate shooter identity. Game outcomes did not differ markedly following wins or losses. D: Distribution of initial velocities over all shooters. Shooters tended to start by moving immediately rightward, toward the goal line, or directly downward. E: Distribution of final vertical positions. Final goalie positions were peaked around 0 and either end of the screen, while shooter final positions were more evenly distributed over the game space. F: Autocorrelation in initial movement and final position across trials. Shooters’ initial movement direction and both players’ final vertical positions exhibited weak correlation across trials, indicative of short-term patterns in strategy. Curves are averages across behavioral sessions. Colors indicate variable.

https://doi.org/10.1371/journal.pcbi.1006895.g002

A generative model of player behavior

Because play in the penalty shot task is fundamentally dynamic and interactive, it is resistant to conventional models based on either discrete action spaces or simple heuristics. Instead, we opted to model player behavior as arising from two pieces: first, a dynamic goal model that encoded each player’s desired onscreen location as a function of both players’ instantaneous positions and velocities; and second, a control model that used these goals as its set points and produced joystick movement (Fig 3). Goals evolved dynamically with the game. This was captured in the model by a “value function” that at each moment drew each player’s desired onscreen location toward areas of high value and away from those of low energy. (This is not the same as the value function in reinforcement learning, which sums all future rewards. Our V is more closely related to policy than expected value). For example, with the goalie at the upper edge of the screen, the energy function for the puck is expected to be low at the lower edge of the screen. However, it is important to note that energy functions for each player were separate. While both made use of explicit information from both players (positions and velocities), each player’s goals were private (i.e., independent of one another given current game state).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Model architecture.

Observable trajectories y_t are generated sequentially from control signals u_t, which in turn derive from goals g_t. In the generative model, evolution of goals is determined by a Gaussian mixture model whose parameters are given by the output of a neural network using state variables s_t as input. The recognition model takes in the lagged history of y_t and returns posterior samples of the goals.

https://doi.org/10.1371/journal.pcbi.1006895.g003

We trained this model using all behavioral sessions of the task as input. To account for variation in play across the experiment, we included a term that increased linearly as a function of session, as well as additional variables encoding experimental conditions (see Methods). Fig 4 shows results from a restricted model using only the last ten sessions of data from a single shooter. Not only is the model able to produce accurate one-step-ahead predictions of the control signal (Fig 4A) and its derivative (Fig 4B), it is able to generate synthetic data that capture the rich behavioral repertoire exhibited by real players (Fig 4C and 4D).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Model fits and generated data.

A: Actual control signals for both coordinates of puck (blue) and bar (red) for a single trial in the holdout set. B: Model predictions for control signals at the next time step given real data up to the current time. The fitted model’s predictions control closely track the real data. C, D: Plots of actual control derivatives and model-predicted derivatives for the same trial. On this scale, deviations become more apparent, but the model still closely fits the data. In both cases, model fits reflect the accuracy of the posterior map, not necessarily the generative model. E: Real puck trajectories (n = 500) drawn from a model fit to the last ten sessions of the experiment (only trials that last less than are displayed, though longer trials were included in training). F: Puck trajectories (n = 500) generated from the model (only trials that last less than are displayed). The model has accurately captured the qualitative features of real play.

https://doi.org/10.1371/journal.pcbi.1006895.g004

To assess the utility of our inferred goals as a construct, we next asked whether these goals contained predictive information about gross strategic behavior. To do so, we used a simple linear regression of final puck vertical position against goal position at each moment of each trial and calculated the coefficient of determination (R²) as a function of time in trial. Fig 5 shows this function time-locked to both the beginning and end of trial. For comparison, we repeated the same analysis using two other predictors, the puck’s instantaneous velocity and the shooter’s eye position. We found that the shooter’s goal became more predictive than the puck’s velocity within the first 0.5s of the trial (Fig 5A) and remained the most predictive of the three variables until the last 0.3s of the trial, when it was overtaken by gaze (Fig 5B). As might be expected, gaze becomes the most predictive single variable late in trial (players attend to the location where the outcome will be determined), while variables like velocity and goal, which only influence position after a lag, become slightly less accurate. Moreover, while state, comprising both the players’ positions and velocities, is more predictive than any single variable, including gaze, adding goal information to state information does improve predictive performance (Fig 5C and 5D). This rough analysis is thus consistent with the idea that our inferred goals provide information not contained in state and can be treated as a latent measure of momentary player intentions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Inferred goals predict game endpoints.

A, B: Predictive power (as measured by R²) for a model predicting final puck position as a function of either shooter gaze position, instantaneous velocity, game state, or inferred goal. C, D: Predictive power for a model predicting final puck position as a function of all variables with or without inferred goals. A, C: Aligned to trial start. B, D: Aligned to trial end. Game states consistently outperform the other variables. Inferred goals are just as predictive as the velocity of movement for the first 0.4s, but become more predictive thereafter. They remain more predictive than either measure until 0.3s before trial end, when gaze becomes a better predictor. In general, including inferred goal states in the model enhances the predictive power.

https://doi.org/10.1371/journal.pcbi.1006895.g005

Model comparison

Our assumption of a Gaussian mixture parameterized by neural networks in Eq 8 is a highly flexible one, raising the question of whether our model simply memorizes trajectories or whether a simpler set of assumptions might do. Fig 6 shows the results of comparing our model to two simpler variants on tests of trial generation and trial completion. The first comparison model assumes the same neural network structure but with a single Gaussian output distribution at each time t: (12)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Model comparison.

A: Difference between actual and predicted control derivatives for a trial (the one in Fig 4) in all three observed dimensions in the proposed model. B: Puck trajectories (n = 100) generated from the proposed model (only trials that last less than are displayed). C: Completed trajectories (n = 100) by the proposed model from of a trial in the holdout set (only trials that last less than are displayed). D, E, F: the same figures based on the model with single Gaussian distribution. G, H, I: the same figures based on the linear model.

https://doi.org/10.1371/journal.pcbi.1006895.g006

The other comparison model uses a linear function in place of a neural network to model the dependency of goals on state: (13) where {(W₁, b₁), (W₂, b₂), (W₃, b₃)} are weights and biases, softplus(x) = log(1 + e^x), and .

We trained all three models on the same dataset with the same hyperparameters and stopping criteria specified above and then compared their performances on fitting control signals and derivatives, generating new data, and completing trials. Our model outperformed the two candidates in all three aspects: the approximate posterior for our model yields the overall smallest prediction error (Fig 6A, 6D and 6G); the generated trajectories resemble the real ones the most closely (Fig 6B, 6E and 6H); and trial completion for the neural network GMM demonstrates the most diversity (Fig 6C, 6F and 6I). That is, the simpler models proved either insufficiently flexible to capture regularities in the data, resulted in quasi-deterministic behavioral policies, or both.

Simulation experiments via generative modeling

By adopting a generative modeling approach along with a flexible class of parameterized functions (neural networks), we have shown that our model can capture the variability present in real data (Fig 4F). An additional benefit of such an approach is that our model can then be used as a simulator for purposes of running controlled experiments. These “counterfactual” simulations allow us to systematically vary task parameters and states to explore the responses of the trained model in configurations only sparsely covered by actual data.

Here, to illustrate the potential of this approach, we performed a simple experiment to explore the effects of goal states on trial outcomes. If, as we claim, goals capture players’ latent targets as a function of changing game state, then altering goals should change subsequent game play. To check this, we fixed a time point midway through a specific trial and performed a series of trial completion experiments (Fig 7). In one set of experiments, the goalie’s goal is fixed to the lower corner of the screen (Fig 7A, red X), while in the other, it is fixed in the upper corner of the screen (Fig 7B, red X). In both cases, the shooter’s goals are allowed to evolve normally, and the goalie’s goals are allowed to move normally after the first time point of the trial completion.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Effects of goal states.

A: Completed puck trajectories (n = 200) for the trial in Fig 6C with the goal state at for goalie is set at the bottom of the screen (only trials that last less than are displayed). B: Completed puck trajectories (n = 200) for the same trial with the goal state at for goalie is set at the top of the screen (only trials that last less than are displayed).

https://doi.org/10.1371/journal.pcbi.1006895.g007

As Fig 7 illustrates, subsequent play is a product not only of initial goals, but of players’ co-evolving control strategies. As one expects, when the goalie’s initial goal is downward, the shooter compensates by moving the puck upward; when the goalie’s initial goal is upward, the shooter steers the puck down. However, what is more interesting to note is the subsequent trial outcomes: in Fig 7A, more extreme puck trajectories result in losses (orange), since these more clearly signal the shooter’s intention and prompt the goalie to reverse direction more quickly. Likewise, in Fig 7B, earlier downturns in puck trajectory also result in losses. In both cases, the initial goal is rapidly reversed from its location at the start of the simulation, while less extreme or more ambiguous trajectories are likelier to result in wins (blue). Thus, the results of our simple experiment suggest both that the model has done more than simply memorizing trajectories and that it can be used to answer “what if” questions via simulation.

Quantifying strategic complexity

Since, as we have argued, players’ inferred goals constitute a moment-by-moment representation of strategic intention, and since the evolution of these goals is governed by each player’s value function V(g, s) (Eq 7), these components of our model are ultimately responsible for capturing the richness of players’ interactions. For instance, at a fixed time t in the trial, the structure of V(g_t, s_t) determines whether a player’s trajectories fall into groups that will split or merge, how variable the trajectories are within each group, and thus how complex the shooter’s corresponding strategy is.

Fig 8A depicts this strategic complexity at a moment midway through the trial. With the puck moving upward, the shooter’s value function (blue) is highest at a cluster of points spread vertically to the left of the goal line, indicating that the goals are pulled in that direction. Likewise, the value function for the goalie (red) is concentrated above the current vertical position of the puck, suggesting a tendency to follow its movement. As the form of these value distributions changes, so does the variability of the resulting trajectories.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Number of change points as a measure of strategic complexity.

A: At a given game state, each player’s energy function (blue, shooter; red, goalie) captures the evolution of his goal in the next time step. Here, the goalie’s energy function is unimodal, while the shooter’s energy function consists of a tightly clustered handful of potential targets. Gray area indicates the visible screen (game area) during the task. White indicates offscreen regions. We use the number of change points in each player’s control signal as a proxy for this complexity. B, C: Histograms of change points for all trials (B) and session means (C) for both players. D: Evolution of strategic complexity across sessions. Over the course of the experiment, the average number of shooters’ change points per trial increased. E, F: Correlation of average number of change points of shooter (E) and goalie (F) with shooter’s win rates. Each dot indicates a single session. Both shooters and goalies benefited from more change points.

https://doi.org/10.1371/journal.pcbi.1006895.g008

To quantify this strategic complexity, we considered the number of direction changes (change points) in each player’s trajectory on each trial. With this definition of complexity, we see that variation in strategic complexity exists at both the trial (Fig 8B) and session level (Fig 8C). A one-way ANOVA shows significant differences across sessions for both shooters(F_119,59764 = 128.950, p < 1.0 × 10⁻⁶) and goalies (F_119,59764 = 216.990, p < 1.0 × 10⁻⁶). Moreover, the average number of shooter change points shows a weakly increasing trend across sessions (Fig 8D; β = 0.00735/session, Wald test p = 4.123 × 10⁻⁸), indicating that players adopted more complex strategies over time. As one would expect, an increase in number of change points, corresponding to an increase in the variability of trajectories, translates to an increase in win rate for both shooters (Fig 8E; β = 4.793%/change point, Wald test p = 3.381 × 10⁻³) and goalies (Fig 8F; β = 3.695%/change point, Wald test p = 8.896 × 10⁻⁴). Thus, players’ strategic complexity, as measured by the number of change points, can serve as a useful metric for evaluating player behavior.

Conclusion

The study of complex strategic decisions, especially social decisions, demands experimental paradigms that replicate this complexity. Here, we have introduced a two-player computerized task, Penalty Shot, that requires real-time behavioral adjustment and facilitates strategic variation. Not only do pairs of rhesus macaques readily learn the game, they exhibit individual playing styles and complex patterns of interactive play. By using a generative modeling approach that leverages approximate Bayesian inference and modern gradient-based training methods, we have shown that it is possible to capture this variability and produce moment-by-moment estimates of latent intentions and strategic complexity suitable for correlation with neural data. These results open new possibilities for the analysis of dynamic behavior and its associated neural data and have implications for the study of social behavior.