## Figures

## Abstract

This paper proposes a simple model to capture the complexity of multilayer systems where their constituent layers affect, and are affected by, each other. The physical layer is a circuit composed by a power source and resistors in parallel. Every individual agent aims at maximizing its own delivered power by adding, removing or keeping the resistors it has; the delivered power is in turn a non-linear function that depends on the other agents’ behavior, its own internal state, its global state perception, the information received from its neighbors via the communication network and a randomized selfishness. We develop an agent-based simulation to analyze the effects of number of agents (system size), communication network topology, communication errors and the minimum power gain that triggers a behavioral change on the system dynamic. Our results show that a wave-like behavior at macro-level (caused by individual changes in the decision layer) can only emerge for a specific system size. The ratio between cooperators and defectors depends on the minimum gain assumed—lower minimal gains lead to less cooperation, and vice-versa. Different communication network topologies imply different levels of power utilization and fairness at the physical layer, and a certain level of error in the communication layer induces more cooperation.

**Citation: **Kühnlenz F, Nardelli PHJ (2016) Dynamics of Complex Systems Built as Coupled Physical, Communication and Decision Layers. PLoS ONE 11(1):
e0145135.
https://doi.org/10.1371/journal.pone.0145135

**Editor: **Ye Wu,
Beijing University of Posts and Telecommunications, CHINA

**Received: **July 16, 2015; **Accepted: **November 26, 2015; **Published: ** January 5, 2016

**Copyright: ** © 2016 Kühnlenz, Nardelli. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

**Data Availability: **This research was carried out using different open-source Python (https://www.python.org) tools including: NumPy (http://www.numpy.org), IPython (http://ipython.org), NetworkX (http://networkx.github.io) and Matplotlib (http://matplotlib.org). The code used to perform the simulation scenarios can be found under https://github.com/tharwan/CoopCPS.

**Funding: **Funding support was provided by Conselho Nacional de Desenvolvimento Científico e Tecnológico (BR)[312146/2012-4] (http://cnpq.br/); the Academy of Finland (http://www.aka.fi/en); the joint funded project A Theory for Sustainable Smart Grids: Combining Communication Theory, Power Systems, Signal Processing and Economics from a Complexity Science Perspective (SUSTAIN), which covers the authors’ salary costs, and the Strategic Research Council/Finnish Academy BC-DC project (n.292854). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests: ** The authors have declared that no competing interests exist.

## Introduction

The modernization of large-scale engineering infrastructures introduces new challenges into their already complicated design [1–3]. For example, electric power grids are large-scale engineering systems built to generate, transmit and distribute electricity from generators to end-users [4–6]. Although their technological development has never stopped, a strong political demand for a structural change is taking place. Such change basically consists in decentralizing generating units (e.g. from nuclear to solar panels and wind turbines), spreading of electric vehicles (which are mobile batteries and loads) and controlling demand based on information technologies; all in all, the traditional consumer is predicted to become a *prosumer*: a consumer who participate more actively in the grid management either by supplying electricity or decreasing their consumption. Modern power grids will become more dynamic and distributed. This will bring new complexities to their already complex dynamic together with new research challenges to cope with them [7].

The same trends—although considered in their own context—can be seen when analysing the modernization of other large-scales systems, from smart cities [8, 9] to factories of the future [10] or the 5-th generation of cellular systems [11]. As in power grids, new complexities in those systems will emerge followed by a need for a new body of knowledge. Notwithstanding the unquestionable technical evolution, there is still a limited number of simple analytic models that are able to capture the dynamics of these modern systems, where the physical infrastructure, the information network and regulations affect, and are affected, by each other dynamics. In this context the present article proposes a discrete-time agent-based model assuming these three layers as constitutive parts of a multilayer system composed by an electric circuit as the physical infrastructure, a communication network where agents exchange local information and a set of regulations that define the agents’ behavior.

The electric circuit is composed by one constant voltage source including its inner resistance and resistors (loads) in parallel [12]. The resistors in parallel are grouped by their controlling agent, as shown in Fig 1. Every agent may add or remove one of the resistors under its control at every time step. One can expect that, the greater the number of active resistors a given agent has, the more power is delivered to it. The actual delivered power is, however, a non-linear, concave, function of the electric current flowing in the circuit; there exists then a saturation point where adding more resistors will decrease the delivered power for the whole system. This behavior is somehow comparable to what happens in real power systems, where overuse leads to a drop in frequency and voltage, reducing the delivered power for all users.

(a) Electrical circuit representing the physical layer of the system. The circuit is composed by a power source *V* and *R*_{V}, and resistors of *R* in parallel, generating a current *I*. These resistors are related to *N* agents that can add, remove or keep the resistors under their control in the circuit. The minimum number of resistors an agent can have is one and there is no maximum. We also consider *N* as the size of the system. (b) The agents are connected in a communication network so that a given agent has access to the information related to the previous action of their first-order neighbours. In the ring topology illustrated here, every agent is connected with two other agents. In this case agent *i* is connected with agents *i* − 1 and *i*+1 with *i* = 1, 2, …, *N*. In the ring topology, agents 1 and *N* are neighbours. (c) At time step *t*, the normalized power delivered *P*_{all}[*t*]/*P*_{typ} to the agents with rising number of resistors *n*[*t*] in the system, where and with *P*_{i}[*t*] given by Eq (1).

Looking at the whole system, a “tragedy of the commons” kind of problem arises [13], where adding a resistor is individually beneficial, while socially harmful. However, in our case, the resource recovers very quickly. The agent’s decision regarding the resistors (adding, subtracting or maintaining) is built upon the following criteria: the behavior of the agent’s neighbors at the last time step, the previous state of the whole system and its own selfishness gene. As we will discuss later, this interactive decision procedure resembles the prisoners dilemma [14–16]. In this case, the agents’ neighborhood is defined by the communication network (where links can be in error), while the selfishness genes of the agents are independent and identically distributed random variables.

### Complexity sciences

Before we start presenting our contribution in more detail, it is worth indicating the theoretical ground that supports our findings: complexity sciences. Complexity is a term used in several diverse research fields [17–19], from theoretic physics to social sciences and biology, to characterize a state that is neither completely deterministic nor random. The so-called complex behavior emerges in systems whose elements interact; they may be heterogeneous and may also adapt their relation rules in accordance to internal and/or external factors.

In his extensive work [20, 21], Wolfram has shown that simple interaction rules applied in one-dimensional cellular automata may lead to unexpected intricate patterns—defined therein as complex—over time. This work tells us that, when looked at a higher level, the spatial-temporal dynamics of a fairly simple deterministic system composed by homogeneous agents that follow fixed interrelation rules may generate complexity. This fact suggests that decentralized systems, based only in local information, might be functional without any controlling entity. In Wolfram’s case the spatial-temporal pattern is determined by the interaction rule. Following his classification, four kind of behaviors can emerge in the one-dimensional automata, namely homogeneous, periodic, chaotic and complex [18, 20].

For some researchers, this fact indicates the system is able to self-organize without any explicit centralized controller. We can cite here few illustrative examples regarded as self-organized [17]: ants working in colonies [22], neurons building a capable brain [17, 23] and birds flying in groups [24] or criticality in plasma turbulence [25]. As an interesting counter point to this perspective, one may argue that the interaction rule that the agents follow is *per se* a kind of central control or a strict regulative force [26]. By using this view, many questions may be posed: *Where do the interaction rules come from? Are they evolving? Are they changing in a much slower time-scale that can be considered as given?* These questions in fact still cause hot philosophical debates among biologists, economists, social scientists and other theorists concerning who controls the “invisible hand” of the system [27].

When dealing with large-scale engineering systems, these questions seem to have clear answers as far as it is designed and follows predefined requirements. This is true for some systems, but it is far from being a universal feature, mainly when the infrastructure is heavily dependent on human actions and interactions. Road networks provide an educative example when one tries to understand the formation of traffic jams [28, 29]. Without going into further details, the key for solving the traffic puzzle is not found by looking at what happens in individual cars or in the design of the whole transportation infrastructure. While these aspects are necessary conditions to the formation of the jam, they are not sufficient to explain the phenomenon. The most accepted theory is built upon the interactions between cars and reactions to the individual behaviors within a specific region of the road network: one car slowing down in highly dense highway causes other nearby cars to slow down as well to avoid collisions, which may trigger a traffic jam that will fade away after some time.

This simple example identifies few important characteristics of complex phenomena: they are spatial and temporal, they never reach stable equilibrium states, the context where the individuals interact and their perception are important to individual decisions, and one individual action might cause a local change that might also trigger changes in the global state of the system. The so-called complexity sciences tries to deal with phenomena that present such characteristics [30].

It is also worth pointing out that simplified models for the power grid (like the “DC Power Flow” [31]) have a long-standing tradition in the literature [32, 33]. While they were initially constructed to cope with available computational power limitation, their use is still widespread nowadays with much more powerful computers [34]. For instance, simplified models for the synchronization in the grid as in [35] are an active focus of the research community (e.g. [36–38]). Such models are able to predict the stability of the network against perturbations, showing ways to optimize its underlying topology [39]. The study of cascade effects [40, 41] to understand the propagation of blackouts and to find the critical links in the grid can be seen as another, more recent, development based on simplified power system models [42, 43]. An agent-based approach (as in the present work) can be also employed to model a wide range of aspects in the power grid, going from electricity market simulations [44] to very specific appliance usage behavior [45] or even the smart grid system as a whole [46].

We follow here the idea of using simplified models to understand the complex behavior that arises in the power grid. However, in contrast to the engineering-focused models as in [47–50], we concentrate more on the higher level interactions across the different layers of the system. Consequently, we have chosen a model that keeps essential features of the system without having a detailed specification of its entities. For this reason, the nature of our work can be seen as “exploratory”, rather than “performance-optimizer” (in the engineering sense) [51].

### Contributions

Motivated by a growing literature in complexity sciences in general and their application in engineering systems in specific, this articles presents a new perspective to analyze multilayered, strongly coupled, systems. We construct a simple, while illustrative, multilayer model composed by agents that control a set of resistors in an electrical circuit. These agents play an evolutionary “prisoners’ dilemma” style of game to decide if they should collaborate or not, based on the local information gathered from their communication network, the estimated state of the whole system and their own random selfishness. Our results indicate that: (i) a wave-like behavior at macro-level spatial-temporal dynamics, which is caused by changes in individual behaviors at the decision layer, can only emerge for a specific system size, (ii) the ratio between cooperators and defectors depends on the minimum gain assumed—lower minimal gains lead to less cooperation and vice-versa, (iii) different communication network topologies—ring, Watt-Strogatz-Graph and Barabasi-Albert-Graph [52]—imply different levels of power utilization and fairness at the physical layer, and (iv) a certain level of error in the communication layer induces more cooperative behavior at the decision layer, affecting the physical layer dynamics in terms of power utilization and fairness accordingly.

## Results

Before starting, we think it is worth describing the key aspects of the proposed model that will help to understand our results. We also systematized some useful notation in Table 1. More details about the multilayer system proposed here can be found in the Methods section. The implementation of our model can be found under [53].

### Agents’ decision process

We assume a discrete-time system such that the changes in the agent behavior occur in time-steps, denoted by . At every time-step *t*, each agent wants to maximize its own power, so their interactions can be then viewed as a round-based game [15]. To achieve that goal, the agent has three options: add a resistor (defecting), remove it (cooperating), or do nothing (ignoring).

To make a decision at time *t*, every agent *i* looks at its gain from the previous strategy *S*_{i}[*t* − 1] in order to decide its new state *S*_{i}[*t*]. The decision process for the agent *i* is the following. If the gain *λ*_{i}[*t* − 1] is greater than or equal to a system-wide pre-defined minimum λ_{min}, the agent sticks to its (successful) strategy at time *t*, i.e. *S*_{i}[*t*] = *S*_{i}[*t* − 1]. If *λ*_{i}[*t*]<λ_{min}, then agent *i* compares its strategy with its neighborhood , which will be defined later in this section. If the majority is cooperative, then and the agent under analysis will also cooperate, leading to *S*_{i}[*t*] = −1. Otherwise, the agent draws a random number between 0 and 1 to be compared to its own selfishness gene *s*_{i} (which is also randomly generated as discussed later) in order to decide whether it will start cooperating. If it does not cooperate, it again draws a random number to be compared to the selfishness gene *s*_{i}, but now to decide if stays inactive (i.e. *S*_{i}[*t*] = 0) or adds another load in the circuit (i.e. *S*_{i}[*t*] = +1). The agent decision procedure is shown in Fig 2.

### Communication network

In the multilayer system proposed here, agent *i* knows the state *S*_{j}[*t* − 1] of the agents through a communication network. We assume that agent *j* always transmits its actual state *S*_{j}[*t*] to agent *i*. The neighborhood of agent *i* is defined as the agents that are directly linked with it. In the case of ring topology, the cardinality of is 2 for all agents . For more complex network topologies, will be characterized differently [52], as discussed later.

The communication links can also experience errors. An error event means the received message by agent *i* contains a different information than agent *j* has sent. Let *S*_{j → i}[*t* − 1] = *S*_{j}[*t* − 1] be the state information transmitted from *j* to *i* at time-step *t* and be the information received by *i*. We consider that error events are independent and identically distributed such that the probability of the event: for all , and , where Pr[⋅] refers to the probability that a given event occurs. It is worth mentioning that the network is a bidirectional graph so that an error event at *i* → *j* does not imply an error event at *j* → *i*, and vice-versa.

If an error event happens, the received information will be uniformly and identically distributed between the other possible states. For example, if *S*_{j → i}[*t* − 1] = −1 and error happens, then or will happen with 50% chance each.

### Physical system

For the physical systems presented in Fig 1, there exists a certain number of resistors that leads to the maximum power gain in the system. If the delivered power is below the maximum on the right, then there will be a gain by removing a resistor until the system has reached such point. Conversely, if it is below on the left, then there will be a gain by adding a resistor.

In this case, one may ask the following question: *Is it possible for the agents to reach the optimal point while fairly delivering power among them (i.e. they indiviadually consume about the same amount of power)?* In the presence of a central control unit this would be a fairly easy problem. First, we need to find the number of resistors that lead to optimal power to then fairly allocate them among the agents by some kind of centralized coordination mechanism. This kind of solution resembles time division schemes in computer networks or cellular systems [54]. For example, if there are ten agents and the optimal number of resistors is twenty, then the central control coordinates the behavior such that all ten agents have two active resistors, summing up to twenty. However, as discussed before, our model does not consider the presence of a central control and the agents have a limited knowledge about other agents.

At time-step *t*, the power each agent consumes *P*_{i}[*t*] with and is given by:
(1)
where , , *a*_{i}[*t*] is the number of active resistors the agent *i* possesses, *r*_{i}[*t*] is the number of active resistors in the system excluding the source resistor *R*_{V} and the ones controlled by agent *i*, and *a*_{avg}[*t*] = (*a*_{i}[*t*]+*r*_{i}[*t*])/*N*.

The physical system is then described by its size *N*, the ratio *μ* of the resistance values and the power source *V*. The resistors are scaled so that the optimal average number of resistors is independent of *N* while the voltage might be scaled with to have a constant ratio of power per agent, as explained later on. The gain that agent *i* experiences at time-step *t* is then defined as:
(2)

This implies that the agents only use the information about the previous time-step *t* − 1. If we expand Eq (2) using Eq (1), the resulting equation that determines *λ*_{i}[*t*] becomes more complicated. To make the analysis clearer, we choose to apply the following approximation (more details in the Methods section):
(3)
such that the gain *λ*_{i}[*t*] is now a function of the variations in agent *i*’s own number of resistors *Δa*_{i}[*t*] = *a*_{i}[*t*] − *a*_{i}[*t* − 1] and in the number of resistors controlled by other agents *Δr*_{i}[*t*] = *r*_{i}[*t*] − *r*_{i}[*t* − 1], as well as the average number of resistors *a*_{avg}[*t*] and the system parameters *N* and *μ*.

### Numerical results

Let be the cooperative agents at time-slot *t*. In this case, where #(⋅) is the cardinality of the set. The spatial-temporal average number of cooperators in the system (*c*_{avg}) is then:
(4)

Fig 3 shows the change of the system behavior with varying size *N*. The left side shows the changes in the spatial-temporal average number of cooperators in the system *c*_{avg} as a function of the system size *N*, while the right side shows a representation of a typical spatial-temporal system behavior for three different sizes *N*. In the latter, each line of vertical pixels represents the state of the system for one time-step: white means cooperation, red means defection and black means doing nothing.

On the left: change in the average cooperation *c*_{avg} depending on the system size *N*. On the right: examples of a typical system behavior as a function of time *t* where the points (pixels) represent the agent state *S*_{i}[*t*] such that red means defection, white cooperation, and black doing nothing for *N* = 1000 (top), *N* = 100 (middle) and *N* = 10 (bottom). The system parameters are: λ_{min} = 0.0005, *R*_{V} = 2Ω, *R*_{0} = *R*/*N* = 200Ω, *p*_{err} = 0.01 and *V* = 1 V. The communication network is configured as a ring.

For small sizes (as when *N* = 10), one can see a kind of checkerboard pattern where cooperators and defectors alternate on time and space axes. For middle-sized systems (as when *N* = 100), the most striking feature is that there exist certain points in time when sudden changes in the behavior happen. The system seems to move closer and closer to a global cooperation state until suddenly it falls back to a state with much less cooperation. Such pattern becomes even more pronounced for a meshed communication network built as a Watts-Strogatz graph [52], as seen in Fig 4. For larger systems (as when *N* = 1000), one can see that a pattern where cooperation is dominant and only few stripes of defecting appear. As we will see next, this behavior is not due to simple scaling effects in the system variables, but rather it results from inherent scaling effects within the physical layer. In other words, certain behaviors can be only observed for certain system sizes *N*. therefore, for a given *N*, one cannot pre-set the system variables expecting a certain kind of behavior. Rather, the size of the system is itself a variable that influences its own macro-level behavior.

On the left: change in the average cooperation *c*_{avg} depending on the system size *N*. On the right: examples of a typical system behavior as a function of time *t* where the points (pixels) represent the agent state *S*_{i}[*t*] such that red means defection, white cooperation, and black = doing nothing for *N* = 1000 (top), *N* = 100 (middle) and *N* = 10 (bottom). The system parameters are: λ_{min} = 0.0005, *R*_{V} = 2Ω, *R*_{0} = *R*/*N* = 200Ω, *p*_{err} = 0.01 and *V* = 1 V. The communication network is configured as Watts-Strogatz graph.

Fig 5 shows how the minimal gain λ_{min} affects the agents’ behavior quantified by average number of cooperators in the system *c*_{avg} for two different system size *N*. One can see that bigger values of λ_{min} lead to more cooperation in the system and different behavior patterns. But only in mid-sized systems (as when *N* = 100) one can see wave-like patterns for certain ranges. We can then infer that the system is dominated by the agent behaviors for high and low thresholds of λ_{min}, while complex behavior only emerges in few cases where a proper interplay between the layers occurs.

Bigger minimum gains leads to more cooperation while smaller ones lead to defection. More complex behavior can be observed for values close to λ_{min} = 0.0001 in mid sized systems. The systems parameters are *R*_{V} = 2Ω, *R*_{0} = *R*/*N* = 200Ω, *p*_{err} = 0.01 and *V* = 1 V. The communication network is configured as a ring.

To have a better understanding on how the minimal gain influences the behavior, we need to analyze Eq (3). We find that the upper limit for the tipping point is given by (more details in Methods section), meaning that after such a point the gain of adding another resistor for a single agent becomes too small. This stands in contrast to the global optimum, which represents the tipping point after which another resistor added to the system as a whole will result in reduced power delivered to the agents. Only in the case that all agents behave equally at every point in time, i.e. , the tipping point for each agent also becomes the global optimum of *a*_{avg}[*t*] = *μ*.

As the system grows, the feedback that each agent can infer is then reduced, so is its influence on the system as a whole. For large systems the deviation of a single agent from the average of the rest of the system has little influence on the average of the whole system. This reduced feedback leads to a situation where agents will become trapped in a state of what we call *cooperative solidarity*. Agents in this trapped state have reduced their number of resistors to minimum (i.e. only one active resistor), while having still not seen a positive gain. Consequently, they remain cooperative as do their neighbours.

In this scenario, all resources of the system will be used by the agents that are not yet trapped, leading to high inequality levels. For large systems the feedback about individual behaviors are so small that the only way to skip from this solidarity trap is, surprisingly, through communication errors. In this case, the trapped agents (wrongly) believe that some neighbours stopped cooperating.

For middle-sized systems, however, the gain may become large enough if a sufficient number of agents synchronize in reducing their number of resistors. This results in a positive gain to all the trapped agents since the system recovers from a state of overusing. This positive gain then leads to agents that no longer seek cooperation as they are already in the lowest state possible and the system has recovered. This in turn explains the wave-like behavior in mid-size systems.

Fig 6 shows the system from a different perspective. Let us first define the power utilization *P*_{util} as the fraction of power that is utilized by the system and the available power:
(5)
where *P*_{i,avg} is the time average power of agent *i* computed as:
(6)

By looking at the power utilization *P*_{util} and the average number of resistors *a*_{avg}, one can see that both are related to each other. Note that all systems in this example are stable in the sense that number of resistors varies within a certain range and does not diverge. This is not necessarily true. Depending on the choices of *μ* and λ_{min} the system can behave differently, e.g. all agents remain with only one resistor, or end up collecting more and more resistors. How close the system can operates to the optimum wildly varies between the different system sizes. The only system that can operate very close to the optimum is the very small system with a size of only five agents (*N* = 5). We also see the sudden jumps in behavior for mid-size systems, while large ones appear to be the most stable.

The small systems can operate much closer to the optimum, while mid-size systems show big jumps after stable periods. The system parameters are λ_{min} = 0.005, *R*_{V} = 2Ω, *R*_{0} = *R*/*N* = 200Ω, *p*_{err} = 0.01 and *V* = 1 V. The communication network is configured as a ring.

To better understand what happens in large systems, we analyze Fig 7, which illustrates the changes in power utilization and inequality (fairness) in power usage between the agents depending on the size of the system *N* and different topologies of communication network. The inequality (fairness) is measured here by the Gini index [55]:
(7)
where *P*_{i,avg} ≤ *P*_{i+1,avg} such that full equality is *G* = 0 and the highest level of inequality is *G* = 1.

The Watt-Strogatz-Graph (WS) setting is: mean degree *K* = 4, rewiring probability *β* = 0.5. Barabasi-Albert-Graph (BA) setting is: number of nodes to attach to *m* = 2,4. The system parameters are: λ_{min} = 0.005, *R*_{V} = 2Ω, *R*_{0} = *R*/*N* = 200Ω, *p*_{err} = 0.01 and *V* = 1 V.

We see that, for small systems, the results are very close together specially for the complex network Watt-Strogatz and Barabasi-Albert [52]. When the system size increases, on the other hand, one can see a growing difference between the results of different topologies, suggesting that the communication layer plays a big role in the system dynamics.

One can also see a drastic change for the Barabasi-Albert network with *m* = 4 that shows a much lower power utilization than all other networks. This is due to the fact that the probability that a node with a degree lower than four exists is very low. This means that, for a given agent breaking free from the solidarity trap, at least four communication errors must happen, instead of two in the ring topology or only one in Watt-Strogatz (for the nodes with a degree of one). therefore, the power is underutilized for the Barabasi-Albert network.

The Gini index analysis indicates, that when the system size grows, few agents receive most of the power, which from a global perspective makes the system very stable. We also see how the outcome is dominated by the structure of the communication network when large systems are considered.

Fig 8 presents more evidence on how the communication system starts dominating the system dynamics when the system size *N* grows. For small systems, one can only see a small reaction with a lot of scattering when rising the communication error probability. Surprisingly we can see global maxima appearing for mid-size and large systems. This fact may indicate that the communication error probability has a similar effect on our systems as temperature on the susceptibility of physical systems [56]. When *p*_{err} = 0, the system can become trapped in a state of solidarity with 100% cooperation (minimal power consumption). With increasing the error probability, the system has a random aspect that allows for agents to defect. However, if the error probability becomes too high, the state information exchange becomes worthless and the system is dominated by the randomness.

The parameters are λ_{min} = 0.005, *R*_{V} = 2Ω, *R*_{0} = *R*/*N* = 200Ω and *V* = 1 V. The communication network is configured as a ring.

## Discussions

We believe that our proposed multilayer system can indeed emulate features of real-world systems with coupled physical, communication and decision layers. Nonetheless, our framework has not been developed to model any specific infra-structure. Our idea is to construct a toy-model where the components are rather simple and easy to understand, but where unpredictable behavior can emerge in certain circumstances, resembling real-world phenomena as modern power grids [5] or cities [9]. As previously mentioned, large-scale systems built upon those layers are getting more and more usual. In any case, we believe that it is also worth discussing the design of the components employed herein. Let us first deal with the rules the agents follow. One might compare the agents to humans or machines acting in behalf of humans. In either case, the possible variety of behaviors may be as large as infinite. The same can be said for the interactions that occur between the agents. In this work, the decision procedure and the communication network connections have been arbitrarily chosen to be understandable and justifiable. Although our simulation assumes the network topology and the decision rules (including the selfishness gene and the minimal power gain) as given, both of them could be evolved as part of the simulation. We could argue that our model allows for complete explanations in the sense that we need not to find explanations that are external to the system; everything can be constructed within the system domain. This, however, might mask the results by, for example, slowing down changes in behavior [26].

For the basic agent principle of maximizing power, it is important to remember that usually power consumption is just a by-product of making life more comfortable and less manual labour intensive by using more loads in the electricity power grid. Similarly to humans, the agents assumed here are most of the time reluctant to remove a resistor once it is installed. They only consider to do so when they did not had a large enough gain from adding a resistor (which means that the system might be overused) or when the social pressure is too much (by being non-cooperative in a cooperative neighbourhood).

To further justify the concept of cooperative solidarity, one has to understand the prisoners-dilemma-type of situations [16]. It is also worth noting that is has been shown that cooperation arises in the evolution of social systems [57, 58]. If the system is close to the optimal point, the agents see a very low gain so that they would have to change their strategy. If most of them reduce their number of resistors, they might receive a little less power (in the case of the system being on the left side of the optimum) or they receive a little more by getting closer to the optimum. However, if only a certain percentage of them reduces their number of resistors while others raise their numbers, the latter gain more power even if the system is very overused. If most of them add resistors, however everyone is worse off than before, which resembles the tragedy of the commons [13].

One of the core findings of this report is that, not only is it possible to create systems with complex behavior through a combination of a few simple parts [20], but indeed every layer influences the systems behavior in nontrivial ways. And in some cases, one layer can even dominate the global system behavior. A special note should be taken in the strong size-dependency of the model, since as far as real-world engineering systems are always subject to changes in the number of users after the initial deployment, leading to unforeseen situations (e.g. power grids [5], cities [9] or highways [28]).

Another interesting aspect can be found in the system behavior in response to communication errors. As was shown in Fig 8, a sharp peak exists for the number of cooperators in the system for a communication error probability of about 1%. From an engineering standpoint, one might prefer a system without errors, which (ideally) leads to a stable and a more predictable behavior. However, the existence of even a small amount of errors leads to a significant change in the behavior and sometimes it might even be desired (e.g. to unfreeze the system from solidarity trap).

Let us assume that the system should work on a state of very high cooperation. Our results indicate that an external attacker trying to disturb the system does not have to shut down the whole communication network to break the dominant cooperative state of the whole system. Rather, it would suffice to generate a small amount of randomly-generated erroneous messages as this will unfreeze the system, which might create a new system dynamic. For an attacker, this would then mean that he does not have to capture the whole communication network to disturb the desired behavior, due to the coupled nature of the system as a whole. Consequently, security precautions should be designed accordingly.

All in all, the present research shall open several options for future work. For example, the inclusion of multiple power sources would be used to simulate a more decentralized power grid topology. The power source could be added as an active agent in the system, having a role of similar to a central controller. It would be also possible to model additional layers like a market where the price for the demanded power (consumed energy) would be set.

## Methods

### Physical layer

The physical layer used in this paper is depict in Fig 1. The value of the resistors *R* that are under the control of the agents should be scaled with the number of agents in the system so that
(8)
where *R*_{0} is a constant value, arbitrarily chosen, independent from the system size.

If all agents have in total *k* appliances, the equivalent resistance of the system is , i.e. *k* resistors of *R* in parallel [12]. The system starts with the different agents having a random number of resistors so that the equivalent resistance is above the system-wide optimal point , computed in terms of power consumed by the agents.

Let us now describe the system from the point of view of a single agent . For agent *i*, all the other individual agents can be combined at time-slot *t* into
(9)
where *r*_{i}[*t*] stands for the number of resistors in the system that do not belong to agent *i*. The agent itself is then described by
(10)

We can then derive the power *P*_{i}[*t*] that agent *i* consumes at time-slot *t* by
(11)
where *V* is the voltage source, *I* is the electric current passing through the source resistor *R*_{V}, and .

Note that we scale the voltage with the square-root of the system size so that the power available for the agents stays constant. We could then write *P*_{i} as
(12)
where *a*_{avg}[*t*] = (*a*_{i}[*t*]+*r*_{i}[*t*])/*N*.

### Communication layer

The communication layer allows for exchange of information between the agents, which is a necessary condition to coordinate their actions in the system. In this case, we need to describe what kind of information is sent by the agents and how they build links creating then their neighbourhood set.

Every agent sends to their neighbours (to be defined next) a message containing its own state in that time step *t* such that *S*_{i}[*t*]∈{−1,0, +1}. The transmitted message may be in error with a given error probability *p*_{err} ∈ [0, 1], which is independent from anything else and uniformly distributed. If an error event happens, the receiving agent sees one of the two other possible states (instead of the transmitted one) with the same probability. The error probability *p*_{err} is the only parameter we control. It is worth mentioning that, although we assume *p*_{err} as given, it in fact is a result of the communication strategy used [54]. Nevertheless, we believe that this more realistic approach goes beyond the focus of this work.

The topology of the communication network defines the neighbourhood set of the agents. In this report we focused on three classes of graphs: ring, Watts-Strogatzs (WS) and Barabasi-Albert (BA) [52]. The ring network is defined in a way that agent *i* is connected with to the agents *i* − 1 and *i*+1 (refer to Fig 1). In this case, agent 1 is connected to agent *N* and vice-versa, so the graph topology resembles a ring

To define more complex neighbourhoods, we employ the Watts-Strogatzs (WS) and the Barabasi-Albert (BA) graphs for social networks. The WS graph is constructed as follows. The graph starts with a regular lattice where each node has exactly *K* neighbours. The links are then rewired with probability *β* resulting in a more random structure.

The BA graph is formed by adding the desired number of nodes step-by-step, starting with a small initial set. Each new node is then preferably connected to nodes with an already high degree (*k*), with probability *p*_{i} such that
(13)
generating then a network whose degree distribution is given by a power law.

### Agent behavior

We explain here the behavior of the agents and its relation to the physical and communication layers previously defined. Let us start in the scenario where none of the neighbours of agent *i*, defined by the neighbourhood set , is cooperative. This is the default state in the beginning. Agent *i* will then behave randomly according to its selfishness gene *s*_{i}. The selfishness gene is a random number with uniform distribution assigned to every agent before start the system simulation itself.

The decision procedure is the following: a random number *ξ* is drawn. If *ξ* > *s*_{i}, agent *i* will switch to the cooperative mode and remove one resistor. Otherwise, it will switch to one of the non-cooperative modes. This means that, when *s*_{i} is big, agent *i* is more selfish. Conversely, agent *i* is more cooperative when *s*_{i} is small

For the non-cooperative modes, the agent will again draw a random number do decide if it will add a resistor if *ξ* < *s*_{i} or do nothing otherwise. The decision process is consistent, but depends on random variables. therefore the higher the selfishness, the higher the probability that an agent starts accumulating resistors.

The agent behavior also depends on the minimal gain λ_{min} that the agents needs to stick to their strategy. We assume that λ_{min} is fixed and pre-defined before the simulation. The gain *λ*_{i}[*t*] that agent *i* has at time-slot *t* is computed as
(14)

Let us now assume that the functions of *t* are continuous so that we can use the total derivative rule as follows:
(15)

Now returning to the discrete domain, we consider *ΔP*_{i}[*t*]≈*dP*_{i}, *Δa*_{i}[*t*]≈*da*_{i}, *Δr*_{i}[*t*]≈*dr*_{i} and *P*_{i}[*t*]≈*P*_{i}[*t* − 1]. Then, the gain *λ*_{i}[*t*] can be evaluated as
(16)

The advantage of proceeding in this way is that one can see that the that the gain not only depends on the amount of resistors agent *i* possesses and the behavior of the system, but also on the system size *N*. The higher the number of resistors agent *i* has, the smaller the first term is since *Δa*_{i}[*t*] can only be −1, 0, or +1. For a very large system (*N* → ∞) and agent *i* non-cooperative (i.e. *Δa*_{i}[*t*] = 1), agent *i* reaches the minimum gain when it reaches
(17)

For example, the minimal gain λ_{min} = 0.0005 used for most of the simulation scenarios leads to *a*_{max} = 2000, which is far above the optimal point .

The second term, which has a negative leading sign, provides feedback. We can then split the term in the feedback from individual actions *Δa*_{i}[*t*] and external actions *Δr*_{i}[*t*]. The negative leading sign means that this term will reduce the gain in case of rising amount of resistors (*Δa*_{i}[*t*] = 1) or will deliver a positive gain in case of cooperation *Δa*_{i}[*t*] = −1 or keep its state by doing nothing *Δa*_{i}[*t*] = 0. In any case, the effect of such feedback will diminish with rising system size *N*.

## Supporting Information

### S1 File. Understanding the feedback of the physical layer on the agents behavior.

https://doi.org/10.1371/journal.pone.0145135.s001

(PDF)

## Acknowledgments

We would like to acknowledge the computing facilities of CSC—IT Center for Science Ltd. (Finland) that was used to run the simulation scenarios.

## Author Contributions

Conceived and designed the experiments: FK PHJN. Performed the experiments: FK. Analyzed the data: FK. Contributed reagents/materials/analysis tools: FK PHJN. Wrote the paper: PHJN FK.

## References

- 1. Helbing D. Globally networked risks and how to respond. Nature. 2013;497(7447):51–59.
- 2.
Nikolic I. Co-evolutionary method for modelling large scale socio-technical systems evolution. TU Delft, Delft University of Technology; 2009.
- 3.
Dam V, et al. Agent-based modelling of socio-technical systems. vol. 9. Springer Science & Business Media; 2012.
- 4.
Kremers EA. Modelling and simulation of electrical energy systems through a complex systems approach using agent-based models. KIT Scientific Publishing; 2013.
- 5. Nardelli PHJ, et al. Models for the modern power grid. The European Physical Journal Special Topics. 2014;223(12):2423–2437.
- 6.
Bush SF. Smart Grid: Communication-enabled Intelligence for the Electric Power Grid. John wiley & sons; 2014.
- 7. Bale CS, Varga L, Foxon TJ. Energy and complexity: New ways forward. Applied Energy. 2015;138:150–159.
- 8.
Batty M. Cities and complexity: understanding cities with cellular automata, agent-based models, and fractals. The MIT press; 2007.
- 9.
Batty M. The new science of cities. The MIT press; 2013.
- 10. Herrmann C, et al. Sustainability in Manufacturing and Factories of the Future. International Journal of Precision Engineering and Manufacturing-Green Technology. 2014;1(4):283–292.
- 11. Hossain E, et al. Evolution toward 5G multi-tier cellular wireless networks: An interference management perspective. IEEE Wireless Commun. 2014;21(3):118–127.
- 12.
Dorf RC, Svoboda JA. Introduction to electric circuits. John Wiley & Sons; 2010.
- 13. Hardin G. The tragedy of the commons. Science. 1968 Dec;162(3859):1243–8.
- 14. Szabó G, Fath G. Evolutionary games on graphs. Physics Reports. 2007;446(4):97–216.
- 15. Archetti M, Scheuring I. Review: Game theory of public goods in one-shot social dilemmas without assortment. Journal of Theoretical Biology. 2012;299:9–20. pmid:21723299
- 16. Gianetto DA, Heydari B. Network Modularity is essential for evolution of cooperation under uncertainty. Scientific Report. 2015;5.
- 17.
Mitchell M. Complexity: A Guided Tour. Oxford University Press; 2009.
- 18. Furtado BA, Sakowski PAM. Complexity: A review of the classics. Policy and Complex Systems. Fall 2014;1(2):3–18.
- 19.
Ott E. Chaos in dynamical systems. Cambridge university press; 2002.
- 20.
Wolfram S. A New Kind of Science. Wolfram Media; 2002.
- 21. Wolfram S. Universality and complexity in cellular automata. Physica D: Nonlinear Phenomena. 1984;10(1):1–35.
- 22.
Wang DQ, Gong QG, Shen XF. An Improved Personnel Evacuation Cellular Automata Model Based on the Ant Colony Optimization Algorithm. In: Applied Mechanics and Materials. vol. 513. Trans Tech Publ; 2014. p. 3287–3291.
- 23. Iarosz K, et al. The influence of connectivity on the firing rate in a neuronal network with electrical and chemical synapses. Physica A: Statistical Mechanics and its Applications. 2012;391(3):819–827.
- 24. Vicsek T, Czirók A, Ben-Jacob E, Cohen I, Shochet O. Novel type of phase transition in a system of self-driven particles. Physical review letters. 1995;75(6):1226.
- 25. dos Santos Lima G, et al. Self-organized criticality in MHD driven plasma edge turbulence. Physics Letters A. 2012;376(5):753–757.
- 26.
Hodgson GM. From pleasure machines to moral communities: An evolutionary economics without homo economicus. University of Chicago Press; 2012.
- 27.
Hodgson GM. Economics and evolution: Bringing life back into economics. University of Michigan Press; 1997.
- 28. Helbing D. Traffic and related self-driven many-particle systems. Reviews of Modern Physics. 2001;73(4):1067.
- 29. Abramson G, Semeshenko V, Iglesias JR. Cooperation and Defection at the Crossroads. PLoS ONE. 2013 04;8(4):e61876. Available from: http://dx.doi.org/10.1371%2Fjournal.pone.0061876.
- 30.
Arthur WB. Complexity and the economy. Oxford University Press; 2014.
- 31. Stott B, Jardim J, Alsac O. DC Power Flow Revisited. IEEE Transactions on Power Systems. 2009;24(3):1290–1300.
- 32. Hahn W. Load studies on the DC calculating table. General Electric Review. 1931;34:444.
- 33.
Casazza J, Ku W. The co-ordinated use of AC and DC network analyzers. In: Proceedings of American Power Conference. vol. 16; 1954.
- 34.
Purchala K, et al. Usefulness of DC Power Flow for Active Power Flow Analysis;.
- 35. Filatrella G, Nielsen AH, Pedersen NF. Analysis of a power grid using a Kuramoto-like model. The European Physical Journal B. 2008;61(4):485–491. Available from: http://www.springerlink.com/index/10.1140/epjb/e2008-00098-8.
- 36. Rohden M, Sorge A, Witthaut D, Timme M. Impact of network topology on synchrony of oscillatory power grids. Chaos: An Interdisciplinary Journal of Nonlinear Science. 2014;24(1):013123. Available from: http://arxiv.org/abs/1305.1634$\delimiter“026E30F$nhttp://scitation.aip.org/content/aip/journal/chaos/24/1/10.1063/1.4865895.
- 37.
Florian D, Bullo F. Synchronization and Transient Stability in Power Networks and Non-Uniform Kuramoto Oscillators. 2010;p. 930–937.
- 38. Nishikawa T, Motter AE. Comparative analysis of existing models for power-grid synchronization. New Journal of Physics. 2015 Jan;17(1):015012. Available from: http://stacks.iop.org/1367-2630/17/i=1/a=015012?key=crossref.64c23e9d1bbe3eaad717d9306453ddf9.
- 39. Menck PJ, et al. How dead ends undermine power grid stability. Nature Communications. 2014 Jun;5. Available from: http://www.nature.com/doifinder/10.1038/ncomms4969.
- 40. Bak P, Tang C, Wiesenfeld K. Self-organized criticality: An explanation of the 1/f noise. Physical review letters. 1987;59(4):381.
- 41. Carreras BA, et al. Evidence for Self-Organized Criticality in a Time Series of Electric Power System Blackouts. IEEE Transactions on Circuits and Systems I: Regular Papers. 2004;51(9):1733–1740. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1333223.
- 42. Holmgren AJ. Using graph models to analyze the vulnerability of electric power networks. Risk analysis: an official publication of the Society for Risk Analysis. 2006;26(4):955–969. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16948688.
- 43.
Huang Z, et al. Modeling cascading failures in smart power grid using interdependent complex networks and percolation theory. 2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA). 2013;p. 1023–1028. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6566517.
- 44.
Krause SM, Boerries S, Bornholdt S. Econophysics of adaptive power markets: When a market does not dampen fluctuations but amplifies them. 2013;012815:5. Available from: http://arxiv.org/abs/1303.2110.
- 45. Kremers E, et al. Emergent synchronisation properties of a refrigerator demand side management system. Applied Energy. 2013;101:709–717.
- 46. Kremers E, Gonzalez De Durana J, Barambones O. Multi-agent modeling for the simulation of a simple smart microgrid. Energy Conversion and Management. 2013;75:643–650.
- 47. Pipattanasomporn M, et al. Multi-Agent Systems in a Distributed Smart Grid: Design and Implementation. Control. 2009;p. 1–8.
- 48. Hernandez L, et al. A multi-agent system architecture for smart grid management and forecasting of energy demand in virtual power plants. IEEE Communications Magazine. 2013;51(1):106–113.
- 49.
Ramchurn SD, et al. Agent-Based Control for Decentralised Demand Side Management in the Smart Grid. 2011;p. 5–12.
- 50. Divenyi D, Dan AM. Agent-based modeling of distributed generation in power system control. IEEE Transactions on Sustainable Energy. 2013;4(4):886–893.
- 51. Schuster P. Models: From exploration to prediction: Bad reputation of modeling in some disciplines results from nebulous goals. Complexity. 2015 Sep;21(1):6–9. Available from: http://arxiv.org/abs/1103.4838 http://dx.doi.org/10.1002/cplx.20234 http://doi.wiley.com/10.1002/cplx.21729.
- 52.
Newman M, Barabási AL, Watts DJ. The Structure and Dynamics of Networks. Princeton University Press; 2006.
- 53.
Python implementation available at: https://github.com/tharwan/CoopCPS
- 54.
Marsic I. Computer networks: Performance and quality of service. Rutgers University; 2010.
- 55. Gini C. Variabilità e mutabilità. Memorie di metodologica statistica. 1912;p. 156.
- 56. Wolf WP. The Ising Model and Real Magnetic Materials. Brazilian Journal of Physics. 2000;30(4):794–810.
- 57. Miranda L, et al. Complex Transition to Cooperative Behavior in a Structured Population Model. PLoS ONE. 2012 06;7(6):e39188. Available from: http://dx.doi.org/10.1371%2Fjournal.pone.0039188. pmid:22761736
- 58. Traag VA, Van Dooren P, De Leenheer P. Dynamical Models Explaining Social Balance and Evolution of Cooperation. PLoS ONE. 2013 04;8(4):e60063. Available from: http://dx.doi.org/10.1371%2Fjournal.pone.0060063.