The authors have declared that no competing interests exist.
Conceived and designed the experiments: SF RLG. Performed the experiments: SF. Analyzed the data: SF. Contributed reagents/materials/analysis tools: SF RLG. Wrote the paper: SF RLG.
Recent theories from complexity science argue that complex dynamics are ubiquitous in social and economic systems. These claims emerge from the analysis of individually simple agents whose collective behavior is surprisingly complicated. However, economists have argued that iterated reasoning–what you think I think you think–will suppress complex dynamics by stabilizing or accelerating convergence to Nash equilibrium. We report stable and efficient periodic behavior in human groups playing the Mod Game, a multiplayer game similar to RockPaperScissors. The game rewards subjects for thinking exactly one step ahead of others in their group. Groups that play this game exhibit cycles that are inconsistent with any fixedpoint solution concept. These cycles are driven by a “hopping” behavior that is consistent with other accounts of iterated reasoning: agents are constrained to about two steps of iterated reasoning and learn an additional onehalf step with each session. If higherorder reasoning can be complicit in complex emergent dynamics, then cyclic and chaotic patterns may be endogenous features of realworld social and economic systems.
When seen at the level of the entire group, the reasoning of many individuals can lead to unexpected collective outcomes, like wise crowds, market equilibrium, or tragedies of the commons. In these cases, people with limited reasoning can converge upon the behavior of rational agents. However, limited reasoning can also reinforce dynamics that do not converge upon a fixed point. We show that bounded iterated reasoning through the reasoning of others can support a stable and profitable collective behavior consistent with the limit cycle regimes of many standard models of game learning.
A limit cycle is a set of points within a closed trajectory, and it is among the simplest nonfixedpoint attractors. Game theorists have been demonstrating the theoretical existence of limit cycle attractors since the 1960s
Should we expect similar complexity in actual human behavior? Humans are capable of “higher” types of reasoning that are absent from most theoretical models, and that have not been empirically implicated in complex dynamics. In work to demonstrate the stabilizing role of iterated reasoning, Selten proved that for a large class of mixedstrategy games, and sufficiently slow learning, adding iterated reasoning to a simple replicator dynamic guarantees the local stability of all Nash equilibria
Cyclic game dynamics have been observed in organisms that are not capable of higherorder reasoning. Animal behavior researchers have described the role of periodic dynamics in resolving coordination conflicts in the producerscrounger problem
When experimentalists entertain dynamic models of human behavior, they tend to treat nonNash behavior as part of the process of eventually converging to Nash
We introduce the Mod Game, an
In the Mod Game,
In our implementation, the maximum integer choice
After all decisions were submitted, all of the round’s choices and earnings were revealed to all players, and the game was repeated for 200 rounds. We also tested a symmetric condition (
Over 22 sessions at Indiana University, 123 psychology undergraduates played in groups of 2–10. The scoring rule does not demand a specific group size, and our design only controlled for group size statistically.
Sessionnumber  Group size  Condition  Time, Date 
1  7  decrement  11∶00, 2011/09/15 
2  3  decrement  12∶00, 2011/09/30 
3  10  increment  13∶00, 2011/09/15 
4  5  increment  11∶00, 2011/09/22 
5  2  decrement  11∶00, 2011/09/23 
6  6  decrement  11∶00, 2011/09/16 
7  9  increment  12∶00, 2011/09/08 
8  8  decrement  12∶00, 2011/09/09 
9  3  increment  12∶00, 2011/09/14 
10  9  decrement  11∶00, 2011/09/09 
11  8  decrement  13∶00, 2011/09/08 
12  7  increment  12∶00, 2011/09/16 
13  2  decrement  10∶00, 2011/09/30 
14  3  increment  11∶00, 2011/09/21 
15  8  decrement  11∶00, 2011/09/08 
16  2  increment  11∶00, 2011/10/07 
17  5  increment  11∶00, 2011/12/05 
18  6  decrement  09∶00, 2011/12/05 
19  8  increment  12∶00, 2011/12/01 
20  3  increment  11∶00, 2011/11/30 
21  3  decrement  12∶00, 2011/11/17 
22  5  decrement  12∶00, 2011/11/10 
Discard 1  8  increment  12∶00, 2011/09/15 
Discard 2  9  increment  15∶00, 2011/12/07 
Discard 3  9  decrement  14∶00, 2011/12/07 
Discard 4  7  increment  10∶00, 2011/12/05 
Discard 5  5  increment  11∶00, 2011/12/01 
Discard 6  4  increment  11∶00, 2011/11/18 
Discard 7  3  decrement  11∶00, 2011/11/16 
Subjects sat at curtained terminals, and interacted with a graphical Javabased interface using the HubNet plugin for NetLogo
You are playing a game with other people. Your goal is to earn as many points as possible. Everyone in your group will choose from a circle of numbered squares 200 times. Your goal is to choose a square that is one more [less] than other people’s squares. The squares wrap around so that the lowest [highest] choice counts as just above the highest [lowest] (like an ace sometimes counts as higher than a king, but still below a two). You get one point for every person who you are above [below] by only one square.
As a bonus, you will be paid for earning as many points as you can. We will pick twenty random rounds and pay you 10 cents per point.
The experiment began after all participants finished reviewing the instructions. Subjects’ 24 choices were arrayed visually in a circle (
This screenshot was taken during a pilot
Though participants were instructed to earn as many points as possible, some exhibited behavior that could not have assisted them towards this end. In particular, some participants repeated their previous round’s choice for large parts of the experiment. Of an original 167 participants, 8 had “streaks” of the same choice for 25 or more rounds in row (1/8 of the total experimental session). In group experiments, individuals influence their group’s behavior, so we cautiously threw away all 8 experiments in which these 8 subjects had participated. The resulting subject pool had 123 participants. The discussion will explore questions of motivation and robustness but, in summary, the results we report are robust to an analysis that includes all 167 participants, and the complete discarded data are available for inspection in
This manuscript reports experimental data from human subjects. Written informed consent was obtained after the nature and possible consequences of the studies were explained. The research contained in this submission was approved by the Indiana University Institutional Review Board.
In games with mixedstrategy Nash equilibria, there is prior experimental evidence for two related but distinct outcomes: a failure to converge to some fixedpoint solution concept (like Nash equilibrium) and a failure to converge to any fixedpoint solution concept. These can be established in a Mod Game with an assortment of complimentary dependent measures. Other methods, like frequency analysis, can then be used towards supporting alternatives to fixedpoint convergence.
We used participant time series–vectors of 200 integers valued 1 through 24–to measure
We introduced
The last two measures gave insight into sequential dependence–how a choice in one round predicts choices in future rounds. While series of random choices should be statistically independent, past experiments in games with intransitive dominance have documented significant sequential dependencies, usually attributed to cognitive or motivational limits
We tested for sequential dependence with analyses of the distributions of first and second differences of participant time series, what we define as
Behavior in the Mod Game will be consistent with uniformly random behavior.
The Mod Game is intransitive in that there is no single action that cannot be dominated by another; the game has no purestrategy Nash equilibrium. For group sizes that are not evenly divisible by twentyfour, and for all of the group sizes we tested, the unique Nash equilibrium is to randomly choose from the 24 choices uniformly. This mixedstrategy equilibrium may seem to be a very naïve null model of actual human behavior–Botazzi and Devetag observe that random play is only rational when others are expected to play randomly
Hypothesis 1 can be rejected by comparing observed values of entropy, efficiency, and clustering to those computed for uniformly random behavior. Though baseline entropy is simple to compute by hand, the other two measures have different baseline values for different group sizes, and simulation was more convenient. If observed values of these measures are significantly different from random benchmark values, Hypothesis 1 can be rejected.
Rejecting Hypothesis 1 would not be particularly provocative. Deviations from uniformly random behavior, which are typical at the individual level anyway, are as likely to result from individual cognitive limits as from convergence upon a higher dimensional attractor.
Behavior in the Mod Game will be consistent with some fixedpoint of a learning dynamic.
This hypothesis can be rejected by looking at sequential dependence. Even if participants do not converge upon uniformly random play, they may have settled upon some other, possibly less principled mixedstrategy. Significant sequential dependence (or a meaningful
Behavior in the Mod Game will be consistent with the convergence of beliefs towards a periodic attractor.
This hypothesis is motivated by observations, in every major class of learning model, of cyclic attractors in games with mixedstrategy equilibria. Supporting Hypotheses 1 or 2 precludes support for Hypothesis 3.
Many high dimensional attractors can exhibit periodicity. While the most common is the limit cycle, this Hypothesis does not specify an attractor, merely that it will have periodic dynamics. Periodicity can be established with Fourier analysis, though it takes statistical methods peculiar to frequency space to distinguish a specific frequency component, or an entire spectrum, from white noise.
The entropy expected from random play was above the 99% confidence interval for observed entropy (
The boxes report means of observed behavior with bootstrapped 99% confidence intervals. The crosses give values expected from uniformly random behavior.
A participant’s behavior in a given round was also dependent on their behavior in the previous round.
The top panel compares distributions over the twentyfour choices, over increment and decrement conditions, against a random baseline. Without temporal information, aggregated choices are difficult to distinguish from uniformly random behavior. The middle panel compares distributions of participant rates. The observed distribution is consistent with the measured mean rate of 4.7 choices per round, forward or backward for increment and decrement conditions, respectively. The bottom panel illustrates accelerations (the difference between consecutive first differences). Observed accelerations are consistent with behavior that either maintains the previous round’s rate or makes only minor adjustments to it. Note that, since the null hypothesis is identical across measures, the circles representing random behavior in each panel have identical radius.
If rate is a meaningful construct in this game, whose strategies are arranged in a circle, then stable rate implies stable periodicity. If participants cycle stably around the strategy set (the circle of choices), any periodicity will show in a Fourier decomposition of their choice sequences. A frequency spectrum may exhibit a larger component at the frequency predicted by the mean rate of rotation.
Since the time series of participants in a group are dependent on each other, data were resampled prior to the frequency analysis. We bootstrapped an independent distribution of observations by randomly selecting one participant’s time series from each of the (statistically independent) groups, and we repeated this sampling procedure 1000 times. Each resulting time series was transformed to the frequency domain with the FFT. Before this operation, missing choices (from the 1.5% of rounds in which an individual made no entry, leaving 24,034 of 24,400 data points) were cautiously replaced with uniform noise from the integer interval {1, …, 24}. Reported spectra and confidence intervals were estimated from this large bootstrapped sample of spectra. The white noise registers artificially low amplitude at frequency zero because of how the data were normalized for transfer to the frequency domain.
We combined data from the
Data were also transformed prior to the frequency analysis. Because of the “jump” between Choices 1 and 24, any cycles around the raw choices describe a sawtooth curve. Sawtooth curves exhibits welldocumented artifacts in frequency spectra, such that a sawtooth with fixed frequency will register many components after decomposition by the Fourier method. To control these artifacts prior to frequency analysis, each time series was transformed to represent the
We then conducted a distributional test in the frequency domain as a preliminary test for periodicity (
The frequency spectra for the first and second 100 rounds of the experiment show the development of cycles. For consistency, the horizontal axis is in units of rate rather than frequency. The frequency spectrum shows a prominent spike in the latter half of the experiment, corresponding to a rate of rotation of about 7 choices per round. This spectrum is the aggregate of spectra from many statistically independent sessions. To control for artifacts and maintain independence, the data were transformed and resampled before transformation to the frequency domain. The dark vertical bar illustrates the spike location predicted by the mean rate. The lighter bars give predictions for mean rates calculated using only the first (left) and second (right) 100 rounds of play.
We complemented the distributional test with a point test for stable periodic behavior at a predicted frequency. This prediction was based on the mean rate of rotation, estimated as the mean of a
As fit to a von Mises distribution, the maximumlikelihood mean rate was 4.7 choices per round, corresponding to a predicted frequency of 0.2 rotations per round. A bootstrapped empirical distribution of the amplitude of the 0.2 frequency component placed it above the amplitude expected from random behavior (mean 1.06, 99% CI [1.04, 1.08], above the amplitude of noise at 0.82).
Video of a typical session gives a subjective associate to the statistical support for periodicity (
We used linear mixed effects to test potential modulators of participant rate. Our model of rate
These tests supported the indifference of rate to group size and condition, and rejected the null hypothesis that rate is indifferent to round (
coefficient  df  LL  χ^{2}  χ^{2} df 


full model  7  −76579  
intercept 

6  −76591  23.9  1  <0.001 
round 

6  −76675  192  1  <0.001 
group size  0.239  6  −76581  2.51  1  0.113 
condition  0.269  6  −76580  0.41  1  0.522 
Since rate is distributed on a circle (with rates of 23 adjacent to rates of 0), the data violate the distributional assumptions of a linear model. For example, the circular von Mises distribution fit a mean rate of 4.7, while the intercept of the linear model
Obviously, these models fit different values of the mean rate







0.239  0.269  



0.422  0.239 



−0.196  −0.266 


− 
−0.925  −0.268 
It is evident in visualizations of both the time and frequency domains that rotations accelerate over time (
The left panel shows the mean rate in each group, at each round. The right panel shows a spectrogram (with a window size of 20 rounds) for resampled observed data. These figures show changes in rotation over the sequence of 200 rounds of play. In the spectrogram, the brightness of a pixel indicates the amplitude of the corresponding frequency component. These panels show statistically significant increases in the rate of periodic behavior, in both the time and frequency domains.
The iterated elimination of nonrationalizable strategies behind Nash equilibrium is intended to mimic the human process of reasoning iteratively through the incentives of other people. By implication, increasing depths of human iterated reasoning are presumed to produce behavior increasingly consistent with Nash equilibria. Behavior in the Mod Game suggests that iterated reasoning may sustain periodic a behavior that does not converge to a fixedpoint.
The entropy, earnings, and clustering of behavior in the Mod Game are inconsistent with the uniformly random play prescribed by the mixed Nash equilibrium, and other popular solution concepts
Going back to Selten, research on iterated reasoning works towards proving that that use of iterated reasoning implies greater fidelity with equilibrium predictions. This cannot be the case if iterated reasoning in the Mod Game is driving periodic behavior. In fact, if trajectories in belief space describe circles around the Nash equilibrium, the prescriptions of iterated reasoning are literally orthogonal to it, and iterated reasoning is complicit in the convergence of sophisticated reasoners towards a periodic attractor.
The heuristic learning direction theory is particularly promising for describing the individual reasoning process behind periodicity and grouplevel clustering. By this theory, participants learn to iterate through a limited
In the Mod Game, participants preferred rates of 1–11 to rates of 13–23 (by a 3∶1 ratio). Why is there such a strong regularity in rates across experimental sessions? There would be no such limit if participants used a theoryfree empirical timeseries method to learn their group’s emergent rate. But rates grounded in iterated reasoning would be expected to show precisely the limits observed. Camerer and Ho fit over one hundred games to an iterated reasoning model and found that a degree of ∼1.5 thinking steps fit the best, and that most games elicit a range of 0–3 steps
Iterated reasoning is an active research subject, but researchers downplay the importance of the heuristic adjustment process that originally accompanied it
Dynamical systems and statistical mechanics offer powerful tools for characterizing the types of complex emergent patterns that we observe here. Intransitive dominance relations between distributed mobile agents have been shown to foster periodic dynamics universally
Generally, a satisfactory model of behavior in the Mod Game will make adjustments around a timedependent rate, inducing nonstationary dynamics through a regime of stable cyclic attractors that captures both the persistent periodicity and the changes in rate over the course of the experiment. Limit cycles are the nonfixedpoint attractors that have received the most attention in game theory, and the observed periodic behavior is qualitatively consistent with this type of dynamic. But periodicity is also consistent with other dynamics, like quasicycles, quasiperiodic oscillations, some chaotic attractors, and even very slow cyclic transients towards a fixed point
Most groups that played the Mod Game can be described as clustering and cycling stably at a slowly increasing rate. Qualitatively, there were some exceptions to the general trend. The middle column of
The description of the subject pool mentions eight participants that made large numbers of repeated choices or null choices and that were excluded from the analysis. Including these participants does not affect the main results of this manuscript: rate of approximately 4, increasing significantly, and driving periodicity that registers a significant spike in a Fourier analysis. The biggest effect of including all of the data is in the polar histogram of rates–the second panel of
Accepting the coexistence of iterated reasoning and periodic behavior does not fix all of the problems presented by this work. Existing models of complex learning dynamics cannot account for important features of periodicity in the Mod Game. If participants’ beliefs are traversing a limit cycle regime, these cycles are different from any that have been predicted. Participants choose their next move using a conception of rate that leads them to “hop” around the circle of choices. As groups, they coordinate their hopping and cluster around specific choices. Neither of these behaviors has been predicted in the dynamics of game learning. Additionally, participants’ rates increase significantly over time, reflecting either convergence, in a nonstationary stochastic system, to a periodic attractor that is changing shape, or the ephemeral behavior of trajectories that are converging only slowly to a stationary periodic attractor.
We have used the Mod Game, an
Cycles in the belief space of learning agents have been predicted for many years, particularly in games with intransitive dominance relations, like Matching Pennies and RockPaperScissors, but experimentalists have only recently started looking to these dynamics for experimental predictions. This work should function to caution experimentalists of the dangers of treating dynamics as ephemeral deviations from a static solution concept. Periodic behavior in the Mod Game, which is stable and efficient, challenges the preconception that coordination mechanisms must converge on equilibria or other fixedpoint solution concepts to be promising for social applications. This behavior also reveals that iterated reasoning and stable highdimensional dynamics can coexist, challenging recent models whose implementation of sophisticated reasoning implies convergence to a fixed point
(PDF)
(MP4)
Contact