Collaboration leads to cooperation on sparse networks

For almost four decades, cooperation has been studied through the lens of the prisoner’s dilemma game, with cooperation modelled as the play of a specific strategy. However, an alternative approach to cooperative behavior has recently been proposed. Known as collaboration, the new approach considers mutualistic strategic choice and can be applied to any game. Here, we bring these approaches together and study the effect of collaboration on cooperative dynamics in the standard prisoner’s dilemma setting. It turns out that, from a baseline of zero cooperation in the absence of collaboration, even relatively rare opportunities to collaborate can support material, and robust, levels of cooperation. This effect is mediated by the interaction structure, such that collaboration leads to greater levels of cooperation when each individual strategically interacts with relatively few other individuals, matching well-known characteristics of human interaction networks. Conversely, collaboratively induced cooperation vanishes from dense networks, thus placing environmental limits on collaboration’s successful role in cooperation.

Pseudocode for the running a single replicate of the main numerical model is given in Alg 1. 1 Each replicate is initiated by setting (and recording) the random seed; importing the graph G and coalitional set S from the graph and coalition library G| e for the given edge density parameter e; generating the initial strategy vector (50% cooperators); and calculating initial payoffs. Subsequently, the main update loop is run T times. This loop involves selecting one coalition (or individual) S to update strategies using the coalitional better response behavioral rule, then re-calculating payoffs. Details on the main steps of the algorithm are given below.

Building the graph and coalition library, G
Prior to running simulations, a graph and coalitional library, G is created which provides for efficient run-time computation.
Denote the set of individuals by N , with |N | = n. A random graph, G(N, E), is constructed with N as its vertex set and E, consisting of unordered pairs of elements of N , as its edge set. At each graph density e and for each of 50 random seeds 2 (seed: 1 . . . 50), G(N, E) is formed by first computing the desired number of edges, (e. n(n−1) 2 ) rounded to the nearest integer, then uniformly selecting this number of edges to form E. In a second step, if any isolated individuals remain, such an individual is selected and an edge added at random to some other individual. This is repeated until no isolated individuals remain. Due to this second step, at very low levels of e, the graph density is slightly higher than e. Note, however, that this second step does not guarantee that G is connected.
Next, for each G in the library, all possible coalitions for coalitional sizes k = 1 . . .k are enumerated and stored together with G in G. Enumeration proceeds iteratively. For k = 1 (individualistic updating), the coalitional set S k=1 is simply N . For k = 2, the set of feasible coalitions is the set of pairs of individuals Algorithm 1 One simulation replicate (rep) for given parameter values Require: T , the maximum number of iterations in a simulation; Γ the game-table; G, the pre-formed graph and coalition library; n = |N |, the number of agents in the population; k, the maximum size of a coalition; p the coalitional updating parameter; e the edge density {initialise:} seed ← Set and record random seed(rep) G, S ← Import graph and coalitions for this graph(G| e , rep) x ← Obtain better response(Γ, G, S, x) π ← Update payoffs(Γ, G, x) t ← t + 1 end while connected by an edge in G, S 2 ≡ E. For k > 2, the algorithm proceeds by first creating a stub set from S k−1 , then, for each stub, generating a new potential coalition by adding a single additional vertex i that is adjacent to any vertex in the stub. The (unique) list of all newly found coalitions is gathered and forms the coalitional set, S k , which forms the new stub set for the next iteration of the algorithm. The algorithm continues until the maximum size of k is reached. Table A provides a tabulation of both graph and coalition size measures for the library G used in this work. For very low e, the number of components is larger than 1. Furthermore, the measured average edge density is slightly larger than e due to the process for avoiding isolated vertices.

Choosing a coalition S ∈ S
At run-time, each update of the model simulation begins by choosing a coalition S from the coalitional library S associated with the given graph G. The size of the coalition chosen is parameterized by the coalitional updating parameter p (see Alg. 1). One could apply a uniform selection over all possible coalitions in S, but as shown in Table A, combinatorics dictates that the number of large (e.g. k = 5) coalitions greatly exceeds the number of small coalitions, rendering a uniform choice over members of S skewed strongly in favour larger coalitions. To avoid this, we first determine a coalition size k, 1 ≤ k ≤k, according to a discrete probability density function over coalition sizes. Specifically, we use the binomial distribution,

Run-time Coalition Formation (large n only)
For combinatorial reasons, when n is large, elucidation of the universe of potential coalitions up to sizē k is computationaly expensive. For large n, we implement a run-time coalition algorithm, replicating the approach to coalition library formation, as follows: Algorithm 2 Run-time coalition formation algorithm Require:k, the maximum size of a coalition; P (k), the probability distribution over discrete coalition sizes 1 . . .k; N , the set of agents in the population; and G, the pre-formed Graph(N, E) for this replicate {initialise:} k ← Choose at random a coalition size(P (k)) i ← Choose at random an agent, i ∈ N S ← i s ← |S| T ← Given G, get adjacent vertices to agents in S, not already in S {main loop (add one other agent at a time):} while (s < k) & (|T | > 0) do j ← Choose at random one agent in T S ← S ∪ j s ← |S| T ← Given G, get adjacent vertices to agents in S, not already in S end while return S 2 Strategic switching: details In the main paper, the fraction of time that individuals played cooperate over the final 500 updates of a 10,000 update run at benchmark conditions, and p = 0.5 is overlaid on the graph structure for differing values of graph density parameter e. We provide here the extant strategic updating dynamics that stands behind these visualisations.
In Figure A we visualise the evolving strategy profile across all 32 individuals and updates 9,501 to 10,000 for each of the three edge density replicates. By inspection, it can be seen that the strategic dynamics comprises common features across each panel: periods of cooperation for two or more individuals are typically created in a coordinated fashion by coalitions (k > 1), whilst defection is either individualistic or enacted by coalitions which comprise a smaller intersection set with a former cooperative coalition. An example of these dynamics is clearly seen playing out for the higher edge density case, e = 0.20 (   Figure 4 of the main paper. In each panel, the changing strategies of each of the 32 individuals are visualised from left to right over updates 9,501 to 10,000. Darker bands indicate periods of contiguous play of Cooperate, whilst lighter bands indicate periods of contiguous play of Defect. To the right of each of the panels A-C the associated interaction graph from Figure 4 of the main paper is reproduced. In Panel C a coalition of three players S = {1, 3, 20} who switch from Defect to Cooperate at update 9,887 is highlighted, together with a subsequent coalition comprising only S = {1, 20} who, at update 9,968 switch back to play Defect, leaving individual 3 playing Cooperate on his own.

Robustness
To establish the robustness of our main results, we run four robustness exercises as follows.

Extreme initial conditions
In our benchmark specification, the population is initialized with 50% of the population playing Cooperate and 50% of the population playing Defect. Here we show that this choice of initial population fraction is not material to our results (fig B).

Disruption to the ability to collaborate
In this experiment, we demonstrate that our main results do not exhibit strong dependence on the history of collaborative events. Running the benchmark specification, as in the main paper, after 3000 periods, we effectively 'turn off' the ability to collaborate in the population, by setting the collaboration parameter, p to 0 from period 3001 until period 4000, following which the parameter is returned to its previous level (i.e. 0, 0.1, 0.5, or 0.9). Results of this experiment are given in Figure C and demonstrate the lack of sensitivity to such an event.

Small-world interaction graphs
It has been argued that typical human interaction graphs of given vertex count n and edge density e are characterised by high local clustering and short average path lengths, placing them between the characteristic features of regular lattices on the one hand, and random graphs on the other (Watts, 1999). Consequently, as a robustness check, we test our main results under benchmark conditions, but with Watts-Strogatz 'small-world' graphs (Watts and Strogatz, 1998) instead of the Erdös-Renyi random graphs considered in the main paper.
Following the same methodology as the main paper, we first create a graph-library and pre-enumerate all possible coalitions up to size k =k. Summary statistics for this library are given in Table C. Next, we repeat the benchmark experiments with the 'small-worlds' graph library and present comparison results in Figure D below (compare to Figure 2 in the main paper). We conclude that cooperation is slightly more prevalent under small-world interaction structures than it is under Erdös-Renyi random graphs over all parameterizations of the model.

Coalitions as cliques
We consider an alternative coalition formation process. Specifically, we reduce the benchmark coalitional library to one in which only coalitions which are cliques (complete subgraphs) are included. The aim is to examine an extreme coalition formation framework where every member of a coalition must have a direct payoff-relevant interaction with every other member of the coalition. As such, we consider this setting a more harsh environment for the development of cooperation as large cliques are more common in dense graphs, which our results have shown are not conducive to the collaborative emergence of cooperation.
We present in Table D below the characteristics of the reduced graph library due to the clique restriction. In Figure E we see that, as expected, cooperation is depressed, relative to the benchmark, under the restriction that coalitions be cliques. However, even in this extreme coalitional setting, significant cooperation arises under non-individualistic strategic updating at low to moderate graph densities.

Large n
All results in the main paper are obtained for n = 32 (following Angus and Newton (2015); Choi and Bowles (2007)), here, we consider an enlarged population, n = 320.
Given the much larger combinatorial space available at this scale, it is not computational practical to elucidate every possible coalition prior to run-time, and instead, run-time coalition formation is implemented as described in detail above (see Section 1.3).
For comparability, two features of the setup need to be adjusted. First, to ensure that agent-wise update frequency is maintained between the small and large n setting, we run the model for 30, 000 updates, 10 times the benchmark used for n = 32. Second, to ensure that payoff trade-offs are isomorphic to the small n case, we re-compute the density parameter, e required to re-produce the equivalent average degree environment produced at the benchmark, n = 32 settings 3 . In Table E the large n graph-library properties are given (compare benchmark case, n = 32, Table A above).  Results of the n = 320 case are given below in Figure F and reflect the benchmark case reported in the main paper.

Costly Coalitions
Finally, one can think coalitional updating as introducing a risk that other agents will not follow through with their commitment to update their strategy in a coordinated manner. Here, one may wish to explore a setting in which agents taking part in coalitional switching must pay a switching, or enforcement fee on doing so. Intuition suggests that such a fee would reduce the likelihood of coalitional switching, at the margin, and may depress coalitional play altogether for a sufficiently high coalitional switching cost.
We explore this setting by appyling a one-time switching cost to the payoff calculation for any agent in a coalition (size k > 1) selected to update. That is, relative to Alg. 1 we adapt the 'Obtain better response' method, such that an additional argument, switching fee, f is added to the calculation of payoffs for each member of the coalition. Each member pays f once, offsetting the expected outcome of a coalitional strategic update to either C or D.
For example, suppose that G = ({1, 2, 3}, {(1, 2), (2, 3)} and all three agents play D. If the two player coalition S = {1, 2} is selected for update and faces coalitional updating cost f , then if 2b − c − f ≥ 0 and 3b − 2c − f ≥ 0 are true respectively for player 1 and 2, a coalitional switch to C would go ahead. Under the benchmark condition, f = 0. Figure G(Panels A-E) gives results from 10 random seeds at benchmark conditions of adding f ∈ {1, 2, 4, 8, 16} to the better response decision rule. For low levels of f , results are similar to the benchmark outcome reported in the main paper, however, as expected, as f grows, the region of long-run cooperation diminishes. For prohibitive levels of f , the apparent regions of cooperation are transient in nature only, since the dynamics of coalitional updating has been slowed dramatically by the high cost of coalitional switching.