Figures
Abstract
A differential evolution algorithm for solving Nash equilibrium in nonlinear continuous games is presented in this paper, called NIDE (Nikaido-Isoda differential evolution). At each generation, parent and child strategy profiles are compared one by one pairwisely, adapting Nikaido-Isoda function as fitness function. In practice, the NE of nonlinear game model with cubic cost function and quadratic demand function is solved, and this method could also be applied to non-concave payoff functions. Moreover, the NIDE is compared with the existing Nash Domination Evolutionary Multiplayer Optimization (NDEMO), the result showed that NIDE was significantly better than NDEMO with less iterations and shorter running time. These numerical examples suggested that the NIDE method is potentially useful.
Citation: He F, Zhang W, Zhang G (2016) A Differential Evolution Algorithm Based on Nikaido-Isoda Function for Solving Nash Equilibrium in Nonlinear Continuous Games. PLoS ONE 11(9): e0161634. https://doi.org/10.1371/journal.pone.0161634
Editor: Wei-Xing Zhou, East China University of Science and Technology, CHINA
Received: April 18, 2016; Accepted: August 9, 2016; Published: September 2, 2016
Copyright: © 2016 He et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are generated from the experiment, the comparable algorithm is described in the paper "Koh, A. An evolutionary algorithm based on Nash Dominance for Equilibrium Problems with Equilibriun Constraints[J]. Applied Soft Computing,2012, 12:161-173." All the codes for our algorithm can be obtained at: https://figshare.com/articles/A_differential_evolution_algorithm_based_on_Nikaido-Isoda_function_for_solving_Nash_equilibrium_in_nonlinear_continuous_games/3574116.
Funding: This manuscript is supported by National Natural Science Foundation of China (71320107003, 71532009). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Nash equilibrium(NE) is one of the most important concepts in game theory, since it reveals the intrinsic link between the game equilibrium and economic equilibrium. NE is widely used in practical applications, therefore, it is a key issue to find NE. However, infinite strategies and complex payoff functions (such as non-concave, non-differentiable, multi-modal, etc.)make it difficult to solve NE. In fact, the difficulty of solving the NE has greatly limited its application in economics, management and sociology.
Economists mainly studied continuous games, in which there are infinite strategies and the payoff functions are also continuous. As we all know, the purpose of game analysis is to predicate a result of a game, which is NE. At present, the commonly used methods for locating NE of a continuous game are the following.
(1) Analytical method is the most straightforward approach, which sets the partial derivatives of every player to zero[1], thus we can obtain the optimum strategy for one player if the other players do not change their strategies. This is also known as player’s best-response function. Solving all the best-response function can obtain the optimal strategy of each player, that is NE[2,3]. The analytical method is useful for providing insights into the behavior of players, however, it is difficult to apply this method to deal with multi-player or non-differential payoff function [4,5].
(2) Mathematical programming is the most widely used method in detecting NE, which is convenient to add certain constraint to the model, and easy to extend to a big system. The mathematical programming method can be divided into the following categories: sensitivity information based [6]; non-interior point algorithm[7]; nonlinear complementarity approach[8], etc. Furthermore, a variational inequality approach [9] has also been applied in Cournot model. However, the mathematical programming method is heavily dependent on the choice of the initial point. If we set an improper initial point, the solution may only be a local optimal solution.
(3) Matrix game is a method which disperses the strategy space, therefore, each player's strategy space is composed of several typical strategies [10]. In addition, the matrix game is also used in the mixed NE[11].
(4) Evolutionary algorithm is a stochastic optimization method, which is not strictly limited to the objective function and can also handling non-differentiable, nonlinear and other complex payoff functions. Son[12] proposed an improved co-evolutionary algorithm—hybrid co-evolutionary programming to find a global optimal NE, which overcame the local optimal shortcoming and improved the convergence of the co-evolutionary algorithm. There are also other optimization algorithms such as invasive weed optimization[13]; cooperative co-evolutionary invasive weed optimization[14]; PSO[15], etc.
In many economic models, there are some special functions to convert NE into a optimization problem, such as Nikaido-Isoda function[16] and Lyapunov function[17]. Moreover, equilibrium also can be described as solution of polynomial equations [18].
When building a game model, economists generally assume that the demand curves are monotonically decreasing functions, and the cost functions are constant or represented by a quadratic functions. The advantage of this assumption is to guarantee a unique NE, which is convenient to find NE. In real economic activity, however, the nonlinear demand functions and cost functions made the model more consistent with the real market competition, this will inevitably lead to the payoff functions become extremely complex, thus making the NE difficult to be solved.
A differential evolution algorithm(DE) for solving Nash equilibrium in nonlinear continuous games is presented in this paper, called NIDE. In each generation, parent and child strategy profiles are compared one by one pairwisely with Nikaido-Isoda as fitness function. In applied research, the NE of nonlinear game model with cubic cost function and quadratic demand function(even some other non-concave payoff functions) are solved. Compared with the existing Nash Domination Evolutionary Multiplayer Optimization (NDEMO), the results suggested that the NIDE was significantly better than NDEMO with less iterations and shorter runtime for solving the NE problems. Meanwhile, it illustrated that the NIDE has high efficiency and wide applicability.
The organization of this paper is as following. Section 2 explains the basic concepts of game theory and provides a searching algorithm based on DE. Section 3 presents numerical examples and compared with NDEMO, and finally conclusions are drawn afterwards
Materials and Methods
Prerequisites
The normal-form game usually consists of three parts:
N = {1,⋯,n} denotes the set of players, n is the number of players;
denotes the strategies set of player i;
ui: S → R denotes the payoff function for each player i ∈ N, where S = S1 × S2 × ⋯ × Sn is a Cartesian product and represents all possible situation of the game, and R is set of real number.
Given a particular topology for a set Si, there is corresponding direct product topology for the set S = S1 × S2 × ⋯ × Sn. For example, if Si is a compact metric space, then S is also a compact metric space. As players’ strategies can be quantified, so the pure strategy of each player Si can be regarded as a bounded and closed convex subset in Euclidean space. If the number of strategies Si for player i is infinite, it is called “Infinite game”.
Definition 1: The infinite game G = {s1, ⋯, sn; μ1, ⋯, μn} is called continuous game, if given a certain topology for a set S, and the payoff function ui: S → R is a continuous function for player i ∈ N.
In an N-player game, when player i chooses his pure strategy si ∈ Si given a pure strategy set of others s−i = (s1, ⋯, si−1,si+1 ⋯, sn) ∈ s−i, he receives a profit of ui(si,s−i). In order to study the equilibrium of continuous game, we need introduce the concept of NE.
Definition 2: In the normal-form game G = {s1, ⋯, sn; μ1, ⋯, μn}, a combined strategy profile is an NE if
(1)
NE is a point where no player can obtain a higher profit by unilateral movement. That is, a profit-maximizing player will not deviate from NE. In fact, if a strategy profile is not NE, that means there must exist a player i who can obtain a higher profit by choosing another strategy
when others choose
, that is
.
Definition 3: In an N-player game, a strategy is called “dominant strategy” for player i, if
(2)
Accordingly, the strategy is called dominated strategy. From the definition 2, if
is a dominant strategy for player i, the outcome resulting from the strategy
is better than the outcome resulting from any strategy
, regardless of what the other players are doing.
Sometimes, a strategy to be a dominant strategy is only relative to the specific strategy, thus we have the following definition.
Definition 4: In a game, a player may have two strategies and
, then strategy
is said to relatively dominant strategy
, than is
(3)
It means that there may exist another better strategy than .
Next, we will introduce the concept of Nikaido-Isoda function[19], which plays an important role in the study of NE.
Definition 5: Let ui be the payoff function of player i, then the Nikaido-Isoda function ψ: (S1 × S2 × ⋯ × Sn) × (S1 × S2 × ⋯ × Sn) → R is
(4)
Each term of the series in the Nikaido-Isoda function represents the change of one player’s gains from strategy xi to yi, while the strategies of the others are held fixed. Nikaido-Isoda function represents the sum of the change in gains or losses of all players.
Based on the definition 4 and definition 5, given two strategy profiles
if ui(yi|x) > ui(x) always hold for each player i ∈ N, we say the strategy profile y is dominant to the strategy profile x. at this point, we have the following proposition hold.
Proposition 1: If the strategy profile y is dominant to the strategy profile x, then
(5)
Prove: If the strategy profile y is dominant to the strategy profile x, then
substitute it into Eq (4), we have
Therefore, ψ(y,x) < ψ(x,y) holds.
If x* is NE, for any y ∈ S, and y ≠ x*, then ψ(x*,y) < ψ(y,x*). Therefore the Nikaido-Isoda function plays the role of a “merit function” measuring the proximity of a strategy to NE.
NIDE algorithm
In 1995, Storn and Price[20–22] proposed a advanced evolutionary algorithm, called differential evolutionary(DE), which was a relatively new stochastic optimization method, and has been shown to be a simple and effective algorithm for the problem of continuous global optimization[21]. The DE requires few control variables, is robust, easy to use and lends itself very well to parallel computation.
There are three important variables in DE: Mutation Factor F, Probability of Crossover CR and Population size NP. Mutation Factor F is positive real number which controls the amplification of the differential variation. Experience has shown that, it is best to choose F between 0.4 and 1.0. In addition, a crossover operation is performed, using the parameter CR, the value of CR is smaller, the robustness of DE will be better. Generally, CR ∈ [0,1]. If the population size NP is larger, the search ability of DE will be intensified, and the calculated amount of DE will be increased. According to literature [21], NP usually to be chosen between 5n–10n, where n is the number of variables.
Based on Proposition 1, by redefinition of selection criteria used in evolutionary methods with the Nikaido-Isoda function, we propose a differential evolution(NIDE) to find the NE. At each generation, parent and child strategy profiles are compared one by one pairwise, using Nikaido-Isoda function to play for the part of fitness function. The whole process can be summarized into the following steps(Algorithm1).
Algorithm1: A differential evolution based on Nikaido-Isoda function(NIDE)
1: Input: the maximum number of iterations Maxit, the population size NP, Mutation Factor F, Probability of Crossover CR, the convergence tolerance ε
2: it ← 1
3: Randomly initialize populations of NP parent strategy profiles P
4: Evaluate fitness to players with P
5: while it < Maxit do
6: for i = 1 to NP do
7:
8: Use Algorithm 2 to create child strategy profiles y
9: if ψ(y,x) < ψ(x,y) then
10:
11: end if
12: end for
13: Randomly choose a chromosome s from P
14: Compute Euclidean norm between s and every member in P
15: if maximum of norm ≤ ε then
16: Terminate
17: else
18: it ← it + 1
19: end if
20: end while
21: Output: Nash Solutions
Child strategy profiles y are created by applying the DE operators via the stochastic combination of randomly chosen parents as discussed in Storn(2003) [23]. Through tested on five different forms of DE operators(DE/rand/1/bin, DE/best/1/bin, DE/rand/2/bin, DE/best/2/bin, DE/rand–to–best/1/bin), we found the operator “DE/rand/1/bin” possessed faster convergence. In order to show the performance of NIDE, we use the same algorithm in literature Koh(2012)[24] to create child strategy profiles, just describe by the following(Algorithm 2).
Algorithm 2: Creating a child vector via DE
1: Input: Current Population P
2: Input: Mutation Factor F, Probability of Crossover CR
3: Input: Lower Bounds LBd and Upper Bounds UBd in each dimension d
4: Randomly choose 3 integers: r1, r2, r3 between 1 and NP
5: r1 ≠ j, r2 ≠ j and r3 ≠ r2 ≠ r1 ≠ j
6: x ← Pj
7:
8:
9:
10: Mutation: Produce a mutant vector z via a stochastic combination of donor vectors
11: z ← a + F(b−c)
12: for d = 1 to D do
13: Crossover
14:
15: Enforce Bound Constraints
16:
17: end for
18: Output: child vector y
In order to check convergence to the NE, randomly select a individual s and compute the Euclidean norm between s and every member of P(lines 13 to 14 of Algorithm1). If the maximum distance is less than ε, the population is judged to have converged to a NE and the algorithm can terminated. Otherwise the process is repeated.
Results and Discussion
In order to test the efficiency of the NIDE, we make a comparison between NIDE and NDEMO[24]. Table 1 gives the parameters used for the numerical experiments just the same in Koh(2012)[24]. All numerical experiments were conducted using Matlab R2013a running on a 32 bit Windows 7 machine with 2 GB of RAM.
Oligopolistic game model with cubic cost function
When building a game model, Economists generally assume that the demand curves are monotonically decreasing functions, and the cost functions are constant or represented by a quadratic functions. The advantage of this assumption is there exists a unique NE, and it is convenient to finding the NE. In real economic activity, however, the nonlinear demand functions and cost functions make the model more consistent with the real market competition, this will inevitably lead to the payoff functions become extremely complex, thus making the NE difficult to be solved.
In microeconomics, however, the total cost function is always non-negative in its definitional domain and decreases to zero on the left side of the inflection point, then it gradually increases while on the right side of the inflection point[25]. The literature [25] pointed out the total cost curve is the analogy of cubic function, so we employ
(6)
There are four forms of cubic function. d > 0 and Δ ≤ 0; d < 0 and Δ ≤ 0; if d < 0 and Δ > 0; d < 0 and Δ > 0 and Δ = 4c2 − 12bd.
The total cost function is a cubic function as a > 0, d > 0, Δ ≤ 0, and the inflection point (,
) falls in the first quadrant, that is,
(7)
Case 1: Example 1 with two players.
Here we first consider the situation with two players i, i = 1,2. Suppose that the output of player i is qi, each player maximizes individual profits from the sale of the homogeneous good. The total outputs are
and the inverse demand function[26] is
Hence the profit function of player i is given by
(8)
We calculate the first order of the profit, i.e.
Based on Eq (9), we can find out the player’s response function,
(10)
It is difficult to obtain the analytical solutions in Eq (10), so we assign a value to each parameter, then use NIDE to detect the NE.
Set
where ai, bi, ci, di submit to the conditions (7).
The NIDE identified the NE(0.7563, 0.7697) successfully. In order to compare the iterations and runtime, the NIDE and NDEMO were both runned 10 times, the results were showed in (Fig 1 and Fig 2). The compare results produced by the NIDE and NDEMO were given in Table 2.
Case 2: Example 1 with three players.
For a more complex case, we consider the three player model. We use the same forms of cost and demand function as in Case 1. Three players maximize their profit by adjusting their strategic variable q1, q2 and q3. The profit functions of each player are analogous to those in (8).
where ai, bi, ci, di submit to the conditions (7).
The NIDE could successfully identify the NE(0.5450, 0.5581, 0.5712)(Fig 3 and Fig 4)as well as Table 3 also show the comparative results of iterations and runtime.
Next, in order to show the high efficiency and wide applicability of NIDE, we introduce two more complex models.
A game model with local NE
This example is taken from [12]. Suppose that there are two players in a market, and the non-concave profit functions for players are defined by
(11)
where pi = 3.1415, and the strategic choice of the two players between 0 and 7.5.
We calculate the first order of the profit, i.e.
Through (Fig 5), we know that the Eq (12) have many solutions(the points of intersection of Fig 5) in the domain [0, 7.5], and it is difficult to differentiate real NE and“local NE traps” use the conventional NE search algorithms.
The NIDE overcomes the local optima and detects the NE which agrees with previous results. The comparative results of iterations and runtime are shown in (Fig 6 and Fig 7) and Table 4.
A game model with 5 players
This example is taken from Koh(2012)[24]. In this example, there are 5 firms, each firm maximizes individual profits from tae sale of the homogeneous good, and inverse demand function is
the total cost curve is
Then, the non-concave profit functions for player i ∈ {1,2,3,4,5} is given by
(13)
The player dependent parameters shown in Table 5.
(Fig 8 and Fig 9) and Table 6 reports results obtained from applying NIDE and NDEMO to this game.
Electricity market with two players.
Electricity market is a classical linear Cournot model[12]. For sample, we assume there are only two players, and they submit bids to generate quantities s1 and s2, respectively.
In order to analyze the strategy of each player, we define each player’s profit by
(14)
where λ = [2750-4(s1+s2)]/75, cost functions C1 = 0.01s12+10 s1, and C2 = 0.01s22+10 s2.
Fig 10 and Fig 11 and Table 7 reports results obtained from applying NIDE and NDEMO to this game.
From Table 7, NIDE’s average running time is 0.2s with only 66 generations of evolution, while NDEMO took 0.5s with 122 generations. It proved the efficiency of NIDE algorithm is still superior to NDEMO in classical liner demand function model.
Conclusions
In this paper, we proposed an evolutionary algorithm for solving NE in nonlinear continuous games. The algorithms of NIDE enabled us to handle the NE problems of nonlinear game model with the cubic cost function and the quadratic demand function, and even other non-concave payoff functions.
In order to assess the performance of NIDE, we compared the NIDE with the existing differential evolution algorithm(NDEMO), comparison of the results showed that the NIDE was significantly better than NDEMO with less iterations and runtime for solving the NE problems. It is more efficient in locating NE in practice and will not result in local optimal.
Meanwhile, through these numerical examples, suggesting that the NIDE could be a potentially useful method.
Acknowledgments
The authors would like to acknowledge Prof. Minqiang Li, Prof. Dominique Guegan in supervising the PhD research; anonymous reviewers of providing helpful feedback and suggestions; National Natural Science Foundation of China (71320107003, 71532009) to provide financial support of this research.
Author Contributions
- Conceptualization: FH GQZ WZ.
- Formal analysis: FH GQZ.
- Investigation: GQZ.
- Methodology: GQZ.
- Project administration: WZ.
- Resources: WZ.
- Software: FH GQZ.
- Supervision: WZ.
- Validation: FH GQZ.
- Visualization: FH.
- Writing – original draft: GQZ.
- Writing – review & editing: FH WZ.
References
- 1. Song Y, Ni Y, Wen F, Hou Z, Wu FF. Conjectural variation based bidding strategy in spot markets: fundamentals and comparison with classical game theoretical bidding strategies[J]. Electric Power Systems Research, 2003, 67(1): 45–51.
- 2. Park J B, Kim J H, Kim J H, Jung M H, Park JK. A continuous strategy game for power transactions analysis in competitive electricity markets[J]. Power Systems, IEEE Transactions on, 2001, 16(4): 847–855.
- 3. Wen FS, David AK. PES News[J]. Power Engineering Review, IEEE, 2001, 21(4): 24–28.
- 4.
Baldick R, Grant R, Kahn EP. Linear supply function equilibrium: generalizations, application, and limitations [M]. Berkeley, CA: University of California Energy Institute, 2000.
- 5.
Baldick R, Hogan WW. Capacity constrained supply function equilibrium models of electricity markets: stability, non-decreasing constraints, and function space iterations [M]. University of California Energy Institute, 2001.
- 6.
Weber JD, Overbye TJ. A two-level optimization problem for analysis of market bidding strategies. Proceeding of the IEEE Power Engineering Society Summer Meeting, Edmondon, Alberta, July 1999, pp: 682~687.
- 7. De LLM, Granville S. The stackelberg equilibrium applied to AC power systems—a non-interior point algorithm. IEEE Transactions on Power Systems, 2003, 18(2): 611~618.
- 8. Wang X, Li YZ, Zhang SH. Oligopolistic equilibrium analysis for electricity markets- a nonlinear complementarity approach. IEEE Transaction on Power Systems, 2004, 19(3): 1348~1355.
- 9. Wei JY, Smeers Y. Spatial oligopolistic electricity models with Cournot generators and regulated transmission prices. Operations Research, 1999, 47(1): 102~112.
- 10. Moitre D. Nash equilibria in competitive electric energy markets[J]. Electric power systems Research, 2002, 60(3): 153–160.
- 11. Lee KH, Baldick R. Tuning of discretization in bimatrix game approach to power system market analysis[J]. Power Systems, IEEE Transactions on, 2003, 18(2): 830–836.
- 12. Son YS, Baldick R. Hybrid coevolutionary programming for Nash equilibrium search in games with local optima[J]. Evolutionary Computation, IEEE Transactions on, 2004, 8(4): 305–315.
- 13.
Sahraei-Ardakani M, Roshanaei M, Rahimi-Kian A, Lucas C. A study of electricity market dynamics using invasive weed colonization optimization [C]// Computational Intelligence and Games, 2008. CIG'08. IEEE Symposium On. IEEE, 2008: 276–282.
- 14.
Hajimirsadeghi H, Ghazanfari A, Rahimi-Kian A, Lucas C. Cooperative coevolutionary invasive weed optimization and its application to Nash equilibrium search in electricity markets[C]// Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on. IEEE, 2009: 1532–1535.
- 15.
[Peydayesh M. A PSO based algorithm for finding global Nash equilibrium[C]//Power and Energy Society General Meeting, 2010 IEEE. IEEE, 2010: 1–6.
- 16. Nikaido H, Isoda K. Note on noncooperative convex games[J]. Pacific Journal of Mathematics, 1955, 5: 807–815.
- 17.
McKelvey R. A Lyapunov Function for Nash Equilibria [R]. Technical Report, California Institute of Technology, 1992.
- 18. Eaves BC, Schmedders K. General equilibrium models and homotopy methods[J]. Journal of Economic Dynamics and Control, 1999, 23(9): 1249–1279.
- 19. Nikaido H, Isoda K. Note on noncooperative convex games[J]. Pacific Journal of Mathematics, 1955, 5: 807–815.
- 20.
Storn R, Price K. Differential evolution-a simple and efficient adaptive scheme for global optimization over continuous spaces[M]. Berkeley: ICSI, 1995.
- 21. Storn R, Price K. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces[J]. Journal of global optimization, 1997, 11(4): 341–359
- 22.
Storn R, Price K. Minimizing the Real Functions of the ICEC'96 Contest by Differential Evolution[C]//International Conference on Evolutionary Computation. 1996: 842–844.
- 23.
Storn R, Price K V, Lampinen J. Differential evolution-a practical approach to global optimization[J]. 2005.
- 24. Koh A. An evolutionary algorithm based on Nash Dominance for Equilibrium Problems with Equilibriun Constraints[J]. Applied Soft Computing,2012, 12:161–173.
- 25. Pu X, Ma J. Complex dynamics and chaos control in nonlinear four-oligopolist game with different expectations[J]. International Journal of Bifurcation and Chaos, 2013, 23(03).
- 26. Zhang YB., Luo XJ, Xue JY. Adaptive dynamic Cournot model of optimizing generating units power output under nonlinear market demand, Proc. CSEE (China)2003, 23, 80–84.